A narrow helper, not a launch
It is 15 November. I have had GPTs and the Assistants API for a year. Today I assemble my own narrow assistant for questions like “how do we run tests here”. This is not an agent on the desktop. This is not a product for sale. This is not a case study at a named client.
My thesis for November 2024 is narrow. An “internal QA assistant” is a GPT or an Assistants thread on a synthetic base: the test process, locator conventions, how to read nightly. The prior art is DevDay on 6 November 2023, not this month. The answer goes through review. The model does not run the suite and is not a quality gate.
Workshop state. The runner is Playwright 1.48.2 from 25 October. The model in the assistant is GPT-4o or gpt-4o-mini. o1-preview since 12 September stays an option, not the pin. .NET 9 has been GA since 12 November; the notes stay on 8, because I do not rewrite the stack in a week. Claude 3.5 from 22 October can do computer use. That is not this post.
What will not be here. There will be no company names and no Confluence dumps. There will be no SaaS premiere. There will be no Pinecone versus Weaviate bake-off. There will be no pin of 1.49.0. There will be a synthetic corpus: one process page, one data-testid convention, one nightly note, three questions and one trap.
Prior art, not an invention
On 6 November 2023 OpenAI showed custom instructions, files, and tools. In the 2023 recap DevDay was a headline of the year, not a procedure. 2024 gave a cheaper model (4o mini from 18 July) and a better chat (GPT-4o from 13 May). That is a cost change. That is not a new category.
In January 2023 I had chat without files. In February a prompt structure. In October 2023 docs from a ticket. The assistant composes those three: instruction, corpus, review. I am not inventing an API.
In January 2024 I wrote that a loop with a goal is not the owner of the suite. An assistant with files is not either. November’s difference is narrow: I do not give the goal “fix nightly”. I ask about a convention I put in myself.
What I put in
The corpus is synthetic and short. A sketch of the process: what is the gate, what is smoke, what is the night. A convention from POM: the locator does not live in the spec, data-testid is an exception, not the default. Three triggers from the CI strategy. A paragraph on how to read a nightly sketch: count, clusters, new versus a known timeout. Zero client tickets.
I do not put secrets in. In November 2023 I wrote that PII and tokens do not go into chat. An assistant corpus is the same chat, only with files. A connection string, a tax id, an invoice, a name from Jira do not enter.
Cursor in the editor is not this assistant. One sentence and stop.
What the assistant is not allowed
I do not run dotnet test from chat as a gate. I do not paste .env. I strike a hallucinated CI job the same way I did in July 2023 triage.
This week’s trap question: “is job api-contract-nightly required on the PR?“. The corpus has no such job. The assistant should say “I do not know”. When it wrote YAML I had not put in, I struck the answer. That is the same reflex as a made-up data-testid in a spec.
Claude computer use from 22 October stays a headline. An agent clicking the IDE does not review my notes. I do not make it daily.
Review like a PR
The answer is a draft. I check the quote against the file. If the sentence is not in the corpus, the assistant should refuse, not guess a process “like other companies”.
Three questions that passed review this week.
First: how I run login locally. The answer quotes the process note and pin 1.48.2. It matches the file.
Second: where load sits. The answer quotes the triggers: nightly or scheduled, not required on the PR. It matches the strategy.
Third: whether I may merge when nightly is red. The answer: the PR gate is separate; nightly does not block the merge by redness alone, but a new login fail goes to a ticket before I pretend the night did not happen. That is process, not an automat.
The trap about the nonexistent job did not pass. That is why the assistant is not a gate.
Summary
In November 2024 a narrow helper on my own notes. Not a launch. Not a RAG shop. Not an assistant at a client.
Three sentences for a year. DevDay 2023 gave instructions and files; 2024 gave a cheaper model. I review the quote against the corpus and strike a job I did not put in. The suite still runs on the runner, not in chat.
What does not follow. It does not follow that I have a product. It does not follow that 1.49 already sits here. It does not follow that computer use drives tests.
Maturity condition: two questions in a row that the corpus does not know end in “I do not know”, not in YAML from imagination.