Multi-file Copilot help - @workspace, not Workspace GA (March 2024)

Two things with almost the same name

It is 15 March, and I have the same problem in every conversation about Copilot: someone says “workspace” and it is not clear which of two things they mean.

The first is Copilot Workspace. It was shown on 8 November 2023 at Universe as a bridge from an issue through a plan to a pull request: you take an issue, get a proposed plan spanning several files, accept it, and a branch is created. GitHub wrote at the time, “when Copilot Workspace is delivered in 2024”, and today that sentence is the full extent of what I know. I have no button, no link, not even a form where I could leave my address and say that I am waiting in line. A keynote demo is not a tool I can turn on, and describing an interface from somebody else’s slides is not writing about my workshop.

The second is @workspace. Lowercase, with an at sign in front, an agent in the Copilot Chat window, listed in GitHub’s changelog on 30 November 2023 alongside @vscode and described two weeks earlier on the VS Code blog. The chat where this agent lives has been generally available in the stable editor since 29 December, which means without Insiders and without the Nightly extension through which I guided readers in May last year. This is the thing I actually have.

My thesis for March 2024 is narrow, and I will defend it throughout this post. Whole-repository context in chat is a real change and I genuinely use it, because questions about a test suite almost never fit in one file. This change concerns what the model can infer, however, not who puts their name on the diff. I get a list of files and a proposed set of changes, review it exactly as I have reviewed a single inline suggestion since April last year, and only then paste any of it into the suite. A bridge from issue to merge did not appear merely because I can ask a broader question.

Here is the state of the workshop today, so that a year from now it will be clear what I mean. The runner is Playwright 1.42.1 from 2 March. I have chat in stable VS Code, on the same subscription I have paid for the gray text in front of the cursor since June 2022. On 27 February, Copilot Enterprise became generally available and brought three things: chat on github.com, knowledge bases, and pull request summaries. In conversations, those three things blur into a single claim that “GitHub already has that agent from Universe”, so I will say it plainly: none of them is Copilot Workspace. The Cursor editor is still not my daily tool.

What will not be here. There will be no claim that today I open Workspace and take an issue to a pull request, because I do not. There will be no tutorial for a tool I do not have in my hands. There will be no move of daily work to another browser tab just because another model was released in March. There will be one synthetic demo from my repository of blog examples: a page object, a spec, and a note about login, one question for @workspace, and two strikeouts.

What multi-file means in March

Gray text works with what it has above the cursor. That was true in technical preview two years ago and it is true today: the input is a file and the output is the next line. When I looked at test boilerplate, I found that neighboring open tabs also entered the context, and that was the full extent of my multi-file support: I opened the component file alongside the test so that the suggestion would stop inventing an API that did not exist in the project. It worked, but the lever was me and my editor tabs.

@workspace reverses that order. I do not guess what to open so that the model can see it. I ask a question, and chat searches the repository itself, selects the fragments it considers relevant, and attaches them to the prompt. An expandable list of the references it used appears above the answer, and this is the most useful interface element in the entire feature: I can see what the model relied on before I read what it inferred. When a file I did not expect appears on the list, that is usually more important information than the answer itself.

I keep the demo in a repository of blog examples, not in a project anyone pays me for. Three files and one business rule:

• pages/LoginPage.ts - a page object with locators in the page class, following the convention I established when refactoring the POM,
• tests/auth/login.spec.ts - one correct-password scenario,
• docs/auth-rules.md - a note saying that a wrong password leaves the user on the form and displays the message “Invalid email or password”.

import { Locator, Page } from '@playwright/test'

export class LoginPage {
  readonly email: Locator
  readonly password: Locator
  readonly submit: Locator

  constructor(private readonly page: Page) {
    this.email = page.getByLabel('Email')
    this.password = page.getByLabel('Password')
    this.submit = page.getByRole('button', { name: 'Sign in' })
  }

  async open() {
    await this.page.goto('/login')
  }

  async signIn(email: string, password: string) {
    await this.email.fill(email)
    await this.password.fill(password)
    await this.submit.click()
  }
}

I ask the first question as reconnaissance, through /explain, and it reads:

@workspace /explain What does the login test suite cover today,
and what is it missing compared with docs/auth-rules.md?

The answer is correct on three points. The suite has one scenario, and it is the happy path. The note describes two behaviors, only one of which has a test. The page object has no method or locator that could read the error message. This is the kind of question where chat performs best, for the same reason I recorded in May 2023: the answer describes files the model has in front of it, so it has nothing to invent. The difference from May is context reach and product status, not magic. Back then I had to open the file in a neighboring tab and point at it; today I only have to ask about it.

It is worth naming what this feature does not do in March 2024, because conversations bundle those things together. It does not run my test or see what the runner printed overnight. It does not know which selectors actually exist in the DOM of the running application, because it indexes the repository, not the live page. And there is no single button that would distribute the answer across three files: below the code block I have copy and insert-at-cursor actions, which operate on one file at a time and are performed by my hand. This last limitation is my ally this month, and the entire next section explains why.

A plan, not a blind apply

I phrase the second question differently. I ask for two things and explicitly nothing more: a list of files that need to change and a proposed diff in the chat window.

@workspace I want to cover the rule from docs/auth-rules.md: a wrong password
leaves the user on the form and displays a message.
List the files to change and the proposed diff. Do not write anything.

I received three suggestions. I accepted one and struck out two, and these are strikeouts from named categories, not cases of “the model did not try hard enough”.

Strikeout one: the wrong file. The proposal added the assertion to the end of the existing correct-password test.

 test('logging in with the correct password opens the dashboard', async ({ page }) => {
   const login = new LoginPage(page)
   await login.open()
   await login.signIn('user@example.test', 'correct-password')
   await expect(page.getByRole('heading', { name: 'Dashboard' })).toBeVisible()
+
+  await login.open()
+  await login.signIn('user@example.test', 'wrong-password')
+  await expect(page.getByText('Invalid email or password')).toBeVisible()
 })

This is the shortest route to green and the worst contract for what this test checks. In its second half, the success scenario starts pretending to be a failure scenario, so when it fails, the report will say “logging in with the correct password opens the dashboard” and will lie in half the cases. With two behaviors in one test, I also lose the cheapest diagnostic information, namely which one broke. The model combined them into one edit not because that is better, but because one edit in one file is the simplest answer to the question “where should I add this”. The question was mine, and the answer is fitted to it.

Strikeout two: an invented export. The diff for the page object contained one line that cannot compile.

 import { Locator, Page } from '@playwright/test'
+import { errorBanner } from './locators'

 export class LoginPage {

There is no pages/locators.ts file in this repository, and it was not on the list of references above the answer, so the model did not read this name from my code. It added the name because, in public code, locators very often live in a separate module, while in my code since August 2021 they live in the page class. This is exactly the category I counted in my April notebook: six out of twenty-nine fixes and rejections had the same reason at the time, namely a selector or identifier that did not exist in the application. Broader context did not eliminate this class of error. It moved the error one level higher, from one line to a plan for three files, and thereby raised the cost of pasting without reading.

That leaves the third suggestion, and it is the only one I accepted as a plan: a new test in login.spec.ts plus a method and locator on the page object, with docs/auth-rules.md untouched because the note is the source of the expectation, not a place for code.

test('logging in with a wrong password leaves the message on the form', async ({ page }) => {
  const login = new LoginPage(page)

  await login.open()
  await login.signIn('user@example.test', 'wrong-password')

  await expect(login.error).toHaveText('Invalid email or password')
  await expect(login.submit).toBeVisible()
})

I added the message locator as getByRole('alert'), and that did not come from chat. @workspace indexed my tests and my note; it did not index the application’s DOM tree and does not know whether that message is in an alert role, in aria-live, or in a toast that disappears after three seconds. I checked it with one run while the page was open, because the rule from last year does not change with broader context: no new locator enters the page object without such a run.

Then I make two runs, not one.

npx playwright test auth/login.spec.ts

The first shows green. The second, after replacing the expected text with nonsense, must show red with a clear message, because a test that cannot fail is worse than no test. Playwright 1.42.1 issues the verdict here, not the chat window, and that is the entire difference between a plan and evidence.

I therefore do not delegate the merge, and this is not an ideological declaration, only a description of who has the information in this matter. Two of the three suggestions had to be discarded, while the one that remained still required one thing from outside the repository: looking at the running application. The person who read those three files signs the plan for those three files today.

Devin as a headline

On 12 March, three days before this post, Cognition announced Devin and described it as an “AI software engineer”: an agent with its own shell, editor, and browser, available for now through early access and sign-ups. This week’s headlines say that teams have just gained a colleague who completes a task on its own.

I do not have access, I have not installed it, and I will not turn this into a tutorial about test maintenance. I record the date, because a year from now it will matter when this conversation began, and one observation about contracts. Devin promises an executor that reaches the end. @workspace promises chat that sees the repository. The first promise requires me to trust a result whose creation I did not see; the second requires me to read text and strike out two thirds of it. In March 2024 I can price only the second, and I do not trade it for the first because the first looks better in a headline.

Claude 3 as a competitor

On 4 March, Anthropic announced the Claude 3 family: Opus and Sonnet from day one, Haiku as an announcement, with the API available without a waitlist. It is a real competitor, just not for what this post is about.

It competes with a browser tab, which is where I paste acceptance criteria and from which I take a draft list of cases. It does not compete with @workspace in the editor, because it has no access to my test repository. To answer the question from the section above, I would have to paste the page object, spec, and note into it, then copy the answer back into the files, which would bring back the context switching that chat in the editor has just removed. This is the same boundary I drew a year ago for chat in Insiders, only there is a more powerful model on the other side today.

I am therefore not moving daily work to Anthropic’s site, and I am not pinning Haiku here as something I have at hand, because in the announcement from 4 March it is announced, not released. Copilot stays in the IDE next to the code; browser chat stays with the draft. The pyramid does not change shape because another engine arrived in March; its shape changes with the decision about what I test and at which level, and none of these models makes that decision for me.

Summary

The multi-file help I have in March 2024 is repository context in Copilot Chat. The product named Copilot Workspace remains a promise from November’s Universe, and I have nothing to write about it beyond the announcement date.

Three sentences I want to be able to read a year from now. On 15 March, I do not open Workspace or take an issue to a pull request; I open chat in stable VS Code, call @workspace, and ask for a list of files plus a diff to read. Of the three suggestions for the synthetic login, I accepted one and struck out two for reasons I know from one line of ghost text: an assertion attached to the happy path and an import from a file that does not exist in the repository. I assembled the only correct plan myself, after the strikeouts and after one run with the page open, and the runner supplied the evidence, not the chat window.

What does not follow from this. It does not follow that Workspace will not be released this year: GitHub announced 2024, and I am recording that date, not negotiating it. It does not follow that Devin has no value, because I have not had it in my hands and have no way to measure it. Finally, it does not follow that @workspace is merely cosmetic, because asking about three files at once really does save me fifteen minutes of searching through directories, and I am not pretending otherwise.

In my 2023 summary, I recorded how I know that a tool has entered the team rather than the hype: it has a named task, it has a place in the process, it has its own row in the pipeline or a conscious reason for not having one, and it has a person accountable for the result. @workspace has all four today. The task is “show me where this rule lives in this repository”, its place is chat while writing a test, it consciously has no row in the pipeline, and I am accountable for the result. Copilot Workspace does not even pass the first point, because I do not yet know exactly what it does.

I define the condition by which I will know that the bridge from an issue through a plan to a pull request has matured as follows: I get a preview I can run myself, and the same issue in two runs produces the same set of files, without an import from a module that does not exist. Until then, multi-file help ends with text in chat for me. A human pastes and signs the diff.