OpenAI's AI discoveries: the verification lesson in Bun

OpenAI published a post about three new results in math, computer science and physics. All of them came from joint work with academic researchers. It wasn't a model spitting out a paper by itself. It was people from the field taking what the AI suggested, pulling it apart and checking every step. I read about OpenAI's discoveries thinking about something far less glamorous than theoretical physics: the PR you're going to open tomorrow with AI-generated code.

The original source is here: Building scientific understanding with academic communities.

What OpenAI announced, without the hype

Here's the honest summary. Language models were used as research partners in three different fields. In each case the model helped reach a result the researchers didn't have before. Then the academics validated everything with the usual methods: reviewed proofs, reproducible experiments, comparison with what was already known.

I won't pretend I understood every technical detail in all three fields. I didn't, and I'm suspicious of anyone who says they got it all in one read. What interests me here is the shape of the work, because it repeats anywhere AI produces something that looks right.

The shape is:

1. The AI proposes.
2. Someone who knows the subject tries to break the proposal.
3. Only what survives becomes a result.

There's nothing new about this. It's the scientific method with a very fast, very confident intern in the middle.

Why OpenAI's discoveries matter for people who code

Because we already live this every day, just without step 2.

You ask the assistant for a function. It compiles. The variable names are good. It even has comments. And then you merge because it "looks right". In science, "looks right" doesn't get published. In our repo, it gets published to production.

AI that gets it right without anyone checking just got lucky, and luck doesn't go in the changelog.

My takeaway from the announcement is that the value didn't come from the model alone. It came from the model plus a community with a high bar. Take away the bar and what's left is a generator of plausible hypotheses. Useful, but dangerous.

Where Bun fits into this story

Bun is a good example of a project that lives on proof, not promises.

When Bun says it's Node compatible, nobody believes it out of politeness. The team runs Node's own test suites against the runtime and tracks what passes and what fails. When it says it's fast, it ships benchmarks anyone can run on their own machine. Sometimes the numbers were challenged, people redid the tests, and the discussion got better because of it.

It's the same cycle as the news: someone makes a claim, the community tries to knock it down, and whatever stays standing earns trust. Bun grew because it accepted being verified in public.

There's a practical side too. bun test ships with the runtime, has a Jest-like API, and runs fast enough that you have no excuse. If verifying is expensive, nobody verifies. If it costs a second, it becomes a habit.

How to verify AI-generated code with Bun

I'll show the trick I use most. It's basically what a mathematician does with a conjecture: compare the clever answer against a dumb answer you know is correct.

Imagine the AI gave you an optimized function to find the maximum contiguous subarray (the famous Kadane):

// max-subarray.ts
export function maxSubarray(nums: number[]): number {
  let best = nums[0]
  let current = nums[0]
  for (let i = 1; i < nums.length; i++) {
    current = Math.max(nums[i], current + nums[i])
    best = Math.max(best, current)
  }
  return best
}

Looks right. But you don't have to trust it. Write the brute force version, O(n²), slow and obvious, and throw thousands of random inputs at both:

// max-subarray.test.ts
import { test, expect } from 'bun:test'
import { maxSubarray } from './max-subarray'

const bruteForce = (nums: number[]) => {
  let best = -Infinity
  for (let i = 0; i < nums.length; i++) {
    let sum = 0
    for (let j = i; j < nums.length; j++) {
      sum += nums[j]
      best = Math.max(best, sum)
    }
  }
  return best
}

const randomArray = () =>
  Array.from({ length: 1 + Math.floor(Math.random() * 30) }, () =>
    Math.floor(Math.random() * 41) - 20
  )

test('matches brute force on 5000 cases', () => {
  for (let k = 0; k < 5000; k++) {
    const nums = randomArray()
    expect(maxSubarray(nums)).toBe(bruteForce(nums))
  }
})

Run it with bun test. If you want to hunt intermittent failures, bun test --rerun-each 20 repeats the suite several times. The brute force is your academic reviewer. It isn't elegant. It just doesn't lie.

Notice this test catches things a visual review won't. Empty arrays, for example: the function above returns undefined for []. My generator never creates an empty array, so the test doesn't catch it either. And that's already a lesson: the verifier has the blind spots of whoever wrote the verifier. That's why science uses peer review, and that's why your test deserves a second pair of eyes.

The obvious objection: "but that takes longer"

It does. About ten extra minutes per critical function.

Now compare that with the cost of a calculation bug in production. I've seen a checkout discount applied twice because a function "looked right". The incident took a whole afternoon, plus the refund, plus the awkward conversation with the client. Ten minutes of testing is cheap next to that.

And you don't need to do this for everything. My personal rule:

• Money, permissions or sensitive data logic: always tested against a reference.
• Parsing and data transformation: tested with random inputs.
• Visual components and API glue: normal review is enough.

AI made the writing part a lot faster. The trusting part still moves at the same pace as before. Anyone who ignores this is just pushing the work onto the Sunday on-call shift.

What actually changes in your workflow

If I had to boil OpenAI's announcement down to one concrete habit change, it would be this: treat AI output as a hypothesis, not an answer.

In practice:

• Ask the AI to generate the test too, but write the dumb reference yourself. If the AI writes both, it can make the same mistake in both.
• Keep bun test running in pre-commit or CI. Verification that depends on remembering doesn't happen.
• When the AI suggests an optimization, ask for the naive version alongside it. You get the oracle for free.

Strong opinion: soon, the skill that separates a good dev from an average one won't be writing code. It will be knowing how to prove the code is right. The researchers behind these three discoveries didn't win by typing faster than the model. They won because they knew where to poke.

My final take

I found the announcement interesting precisely because it doesn't sell AI as a lone genius. The credit goes to the process: the model proposing, competent people verifying, a community open to challenge. It's the same process that earned Bun its credibility, and the same one that should protect your repo.

AI will keep getting smarter. Your test suite needs to keep up. If you want to see how I apply this in real projects, take a look at my projects.

AI didn't discover anything on its own, and that's exactly the point.

OpenAI published three new results in math, computer science and physics. In every one, the model proposed and real researchers tried to knock down each step.

In science, "looks right" doesn't get published. In our repo, it gets published to production.

My habit today: AI code is a hypothesis, not an answer. I compare the optimized version against a slow, obvious brute force on thousands of random cases with `bun test`. The brute force is ugly, but it doesn't lie.

Soon, what separates a good dev from an average one won't be writing code. It will be knowing how to prove it's right.

I wrote the step by step with an example on the blog. If it makes sense for your team, the link is in the comments.

#ArtificialIntelligence #Bun #TypeScript #Testing #SoftwareDevelopment