DeepMind Double-Blind Gemini Eval 2026: What the Enclave Pilot Means
DeepMind piloted a sealed eval of Gemini 2.5 Flash Lite so neither side sees the other’s secrets. The news is the process, not a leaderboard row.
On 27 August 2026, Google DeepMind said it evaluated Gemini 2.5 Flash Lite with Singapore AISI, OpenMined, MLCommons and AVERI inside a sealed enclave: the lab does not see the items, the testers do not see the weights.
This is not another leaderboard drop. Coverage in TechTimes and The New Stack, drawing on the technical report, treats double-blind evaluation as a structural fix for a trust deadlock.
Why zero-logging was never enough
If a public benchmark leaks into the training corpus, the score measures retrieval. DeepMind cites earlier work that found leakage signs in roughly half of 31 models, and 2026 research that contamination can inflate scores, especially for larger systems.
Vendor APIs expose prompts. Sharing weights exposes the asset. Zero-logging is a promise. The July 2026 Singapore Consensus on Global AI Safety Research Priorities listed double-blind infrastructure—evaluator cannot see parameters, developer does not know the exact items—as a priority gap.
API evals
Prompts traverse the vendor’s stack, so the lab can see the questions.
Weight handoff
Evaluators gain reproducibility; the lab loses control of a costly artifact.
Contractual zero-log
Good-faith compliance still fails on caches, mistakes, or a later training run.
What happens inside the enclave
-
1
Remote attestation
Both sides check that the guest is the agreed software, not a build with an exfil path.
-
2
Encrypted load
DeepMind loads Gemini 2.5 Flash Lite weights and inference code; evaluators load hidden prompts and scoring code. Weights sit in H100 encrypted GPU memory; prompts sit in Intel TDX-encrypted host memory.
-
3
Bounded results only
The evaluator receives only allowed outputs. The guest is torn down. The report puts crypto overhead under about five percent.
Software controls and leftover trust
Encryption is not enough if the guest can call home. PySyft requires joint code review and blocks sensitive egress. In December 2024 OpenMined ran an H100 enclave pilot with UK AISI and Anthropic on an open-source proxy against CAMEL-bio. In March 2026 NIST’s Center for AI Standards and Innovation signed a research agreement with OpenMined to use PySyft for secure evals.
Trust moves rather than disappears: from a bilateral contract to NVIDIA and Intel chips, and to Google Cloud services that sign attestation reports.
| Role | Usual path | This pilot |
|---|---|---|
| Model owner | Sees API prompts or signs a zero-log clause | Cannot read items inside the guest |
| Evaluator | Needs weights or must trust the lab | No weights; bounded results only |
| Model class | Any size in principle | Systems that fit one GPU first |
MLCommons also notes that hiding items does not make a benchmark well designed. Check scores in the primary report; this piece does not reprint unverified leaderboard numbers.
- Legal friction
- DeepMind’s report lists inter-org agreements and joint code review as larger scaling hurdles than the GPU bill.
- Multi-GPU gap
- Researchers are working on encrypted H100/B200 fabrics for distributed inference; that is not a shipping eval tool yet.
When a team needs one picture
After the paper, groups usually need a shared sketch of who attests, who supplies silicon, and which layer is still trust. You do not need another meeting suite. Open a short space on tidemeet, or read sessions without accounts and how to create a space.
# walkthrough
attestation → weights-in → prompts-in → bounded-resultsQuestions
Does this make benchmark scores trustworthy?
No. It closes one failure mode: each side having to see the other’s secret. Benchmark quality, post-run item hygiene, and the largest flagship models sit outside this single-GPU pilot.
What did Gemini score?
Public write-ups stress the method, not a number. Use the DeepMind and partner technical report; do not treat unverified ranks as the result.
Can other labs copy this tomorrow?
Confidential computing is not Google-only. Contracts, joint review, and attestation roots still lean on a cloud and two chip vendors. Scaling on non-Google infrastructure will add friction.
Why mention a temporary canvas?
There is no product tie-in to the eval. If you only need a diagram of the trust boundary, temporary meeting tools explains when a short browser space is enough.