I joined the waitlist on 22 March, today I have chat in my editor
On 22 March, GitHub announced Copilot X and released a form alongside the announcement. I filled it in that same day because one thing in the entire announcement interested me: chat that sits in the editor and can see the file I am working on. Then nothing happened for several weeks, until I received an email in May saying that I had access.
I am writing this post the day after I first opened that window next to a real test file. I do not have a month of work behind me, I have no opinion on how it works in a team, and I have no idea when it will leave preview. I have a few evenings, one synthetic NUnit file, and a list of things I had to cross out of its answers.
I want to make this clear immediately because I can see the distinction starting to blur in conversations: Copilot Chat in May 2023 is not a product that everyone gets. It is a technical preview behind a waitlist. It is not in the pricing, it is not in stable VS Code, and there is no switch in the settings that enables it. Ghost text, the gray text in front of the cursor, is a completely different matter: it left technical preview almost a year ago and I have used it every day since. Combining these two things into one sentence saying that “Copilot now has chat” is simply untrue.
There is a second boundary I want to establish at the start. Since January, I have been drafting test cases in a browser window, and that has not changed. Browser chat cannot see my repository. Whenever I want to ask about a specific helper, I have to copy it there together with some context and then copy the answer back. The entire difference described in this post fits into one sentence: I can now ask a question about this file in the same window where I have that file open.
The setup that works for me in May
The May setup is narrowly defined, and there is no “install the extension and you are done” option. I am listing exactly what I have because in six months no one will be able to reconstruct what it looked like at the start.
First, an active Copilot subscription. This is the same subscription as the one for ghost text, costing 10 dollars per month or 100 dollars per year. The subscription alone does not provide chat, and this is the most common misunderstanding I see.
Second, the same GitHub ID I used to join the waitlist. Joining the waitlist with one account and signing in to the editor with another is the easiest way to install the extension correctly, see no icon, and have no idea why. I found this out myself because I have a separate account for experiments.
Third, VS Code Insiders. Not stable VS Code. Insiders is a separate installation with a green logo, its own settings directory, and its own set of extensions. It runs alongside the stable release without conflicting with it. It updates every day, so it is not the environment where I keep everything I work on.
Fourth, the GitHub Copilot Nightly extension. I install it from the marketplace in Insiders and disable the regular Copilot extension in that profile because the two interfere with each other. Then I fully restart the window, not merely reload it. After the restart, a chat icon appears in the bar on the left and the conversation view appears in the command palette.
One side effect of this setup is that I have two editors. In stable VS Code, I write what goes into the repository and only have ghost text. In Insiders, I have chat, and that is where I open anything I want to ask a question about. I do not recommend this as a long-term way to organize work. This is what life with a preview looks like, and I will not pretend it is convenient.
One note about versions, so there is no doubt about what I tested. The file in this post is NUnit on .NET 7. In another repository, I have end-to-end tests using Playwright 1.33.0, and I ask exactly the same kinds of questions there, but the answers are weaker, as I will explain shortly.
What Chat sees and what it does not
This is the section I wish I had read before I started, because the first disappointment comes from the false assumption that something on the other side has read my project.
Chat sees the active file and whatever I select in it. If I select a method and ask “what does this do,” the question concerns that method and I do not have to paste it anywhere. It also sees diagnostics from the editor, meaning the same red underlines and error codes that I have on the screen. This is the part browser chat does not have and that cannot be worked around by pasting, because a compilation error without the rest of the context looks like a puzzle.
It does not see the entire repository. The class I am testing lives in another project in the same solution, and until I open it in an adjacent tab and point it out, answers about it are guesses based on its name. It does not see the git history, it does not see the results of the latest run, and it does not know that one particular test fails every tenth time on the agent.
It also sees none of my secrets, and that is a deliberate decision on my part. I do not have an .env file open when I work with chat, and I do not paste connection strings or API keys into the conversation. This is not because I have evidence that something bad happens to them. It is because this is an external service in preview, the terms may change, and pasting someone else’s secret into an external service is not a decision to make quietly. It is the same rule I wrote down during my first API experiments in 2021: nothing from production, no real data, and no fragment of a dump.
Three questions I ask about a test
The file is invented for this post and concerns the same nonexistent feature I used for the model comparison in March: a discount code in a shopping cart. It does not contain a single line from any project I work on.
[TestFixture]
public class VoucherTests
{
private const decimal MinimumTotal = 100.0;
private static Cart CartWith(decimal total, string currency = "PLN") =>
new Cart { Total = total, Currency = currency };
private static Voucher Discount(string code, DateTime expiresAt) =>
new Voucher { Code = code, MinimumTotal = MinimumTotal, ExpiresAt = expiresAt };
[TestCase(99.99, false)]
[TestCase(100.00, true)]
public void Applies_only_above_minimum(decimal total, bool expected)
{
var service = new VoucherService(new FixedClock(new DateTime(2023, 5, 15)));
var applied = service.TryApply(CartWith(total), Discount("SPRING", new DateTime(2023, 6, 1)));
Assert.That(applied, Is.EqualTo(expected));
}
}Question one: “explain what these two helpers do and what is not visible in this file.” This is where chat performs best, and not because it is intelligent. It is because the answer describes the code in front of it, so there is little room to make things up. I received an accurate description of both methods, a note that Discount hardcodes the minimum so no test can check behavior with a different threshold, and a sentence pointing out that the expiration date is always in the future relative to the clock, so the expired voucher branch is never executed here. Three good points, two of which I would have noticed myself and one, the threshold, which I had missed for two days.
I crossed out one sentence. Chat added that the default PLN currency “matches the store’s currency,” which is not in this file and not anywhere in my code. This is the same class of error as an invented selector: the model takes a plausible-sounding business justification and puts it into the description as if it had read it somewhere.
Question two: “write a test for the expired voucher branch, following the convention in this file.” Here I crossed out more than I kept. The proposal used DateTime.Now instead of the injected clock, exactly what the rest of the file is designed to avoid, and would also have turned the test into a time bomb for the day after expiration. Crossed out. It called service.Validate(...), a method that does not exist in this class because only TryApply exists. Crossed out. It named the test TestExpiredVoucher, even though the underscore convention is right in front of it. Crossed out.
One thing remained, and that one thing made it worthwhile: the proposal included a case exactly on the boundary, with a timestamp equal to the expiration time, and asked in a comment whether the voucher should still be valid at that moment or should already be invalid. My invented specification does not answer that question, and this is exactly the sort of gap to take to an analyst. The net result: I received a draft of which I kept perhaps one third, plus one question I would not have asked myself.
Question three: “what does this compilation error mean?” The first line of my fixture is wrong, and I left it that way deliberately because it is the typo I make most often in code involving money. private const decimal MinimumTotal = 100.0; produces CS0664, which means that a double literal cannot be implicitly converted to decimal. I selected the line, asked the question, and received the correct answer together with the 100.0m fix. I did not have to copy the error message into a browser or explain what type it concerned. This is the moment when chat in the editor genuinely beats the adjacent tab.
I crossed out the second part of the answer. Chat suggested changing the field type to double as an alternative. Formally, that removes the error. In practice, it is the worst advice you can give for code that calculates monetary amounts, and if I clicked “insert” instead of reading it, I would have introduced a floating-point type for money into the test project.
The questions in the Playwright repository looked the same, but there were fewer good answers. The context there lives in page objects in other files and in configuration that chat cannot see, so the answers become generic. This clearly shows what the entire feature rests on: a single open file.
What I do not delegate
I already described my first two weeks with ghost text in the post about my first test boilerplate, and the conclusion from that post carries over in full: what I receive is a draft, and the review is mine. Having a conversation changes nothing in that agreement, except that the answers are longer, sound more confident, and are easier to paste without reading. With gray text, I assess one line and see it in context. With chat, I receive twenty lines in a separate window, where I have to manage the context myself.
I do not delegate the merge decision. Code produced by chat goes through the same review as any other code, and the pull request description does not say “generated, so probably good.”
I do not delegate the assessment of a flaky test. Chat cannot see the last twenty runs on the agent, does not know that the test fails only when run in parallel, and anything it says on the subject is a guess based on the contents of the file.
I do not delegate the decision about whether a test belongs in CI at all. That is a question about runtime, maintenance cost, and whether anyone will respond to a red result. None of those three things are in the open file.
I am not giving up ghost text either. It stays where it was, in the stable editor, when I write the next line. Chat is for questions that previously made me open a browser, not a replacement for completion.
And I do not rewrite what I recorded earlier. The post about Copilot’s technical preview describes February 2022 and the state of knowledge in February 2022. Last month’s conclusions about reading suggestions stay as they were. Adding today’s knowledge to the archive only makes it impossible to reconstruct a year from now what was known and when.
Summary
After a few evenings, I have one conclusion I can defend: chat in the editor reduces context switching and nothing else.
I will break this down into three sentences. I can ask about the open file without copying it into a browser and without copying the answer back, and this makes a substantial difference with a compilation error because the complete message and the complete line are already available on the other side. Answer quality declines precisely when the answer requires something outside the open file, so the benefit is smaller in a repository where context is split up than in a single fixture. Fabrication has not disappeared at all: across three questions, I crossed out a nonexistent method, an invented business justification, the wrong naming convention, DateTime.Now in place of an injected clock, and advice to calculate money using double.
What does not follow from this? It does not mean I have a new tool in the team’s process, because half the team does not have access and the other half will not set up a second editor just to ask a question. It does not mean this is a finished product, because it is a preview behind a waitlist, with a nightly extension and an editor version that changes every day. Finally, it does not mean everyone paying for Copilot has it, because they do not.
What will I return to in a few months? I want to see how this chat handles context split across several files because that is the actual shape of every test project I run, and today it is the chat’s weakest point. I also want to see how many items on my list of things to cross out disappear on their own and how many are inherent to a tool that generates plausible text. My bet is that the strike-outs will remain and it will still be worthwhile, but they need to be counted as part of the cost, not treated as a surprise.