Case Study: Using Verifiable AI for LLM as a Judge

As the market continues to explore practical use cases for TEE-based LLMs, Exponent AI and Marlin offer a compelling example. By leveraging Marlin’s Trusted Execution Environment (TEE) platform, Oyster, to power Exponent’s Vector3 Network, we demonstrate how trustless and secure computation can elevate AI capabilities. This collaboration highlights a key real-world application: running LLM as a judge model in a decentralized, verifiable, and collusion-resistant manner, delivering trust and transparency to AI x Web3 systems.

About Marlin Oyster

Marlin Oyster is a TEE-based platform designed to provide secure and tamper-proof execution environments. By isolating applications from the host operating system, Oyster ensures confidentiality and integrity for sensitive computations. Trusted by leading projects across industries, it’s tailored for applications that demand verifiable operations, like those at the intersection of AI and Web3.

Key Features of Oyster:

• Confidentiality: Protects data during computation, ensuring that private keys and sensitive information remain secure.
• Integrity: Ensures that programs execute exactly as intended, safeguarding against unauthorized modifications.
• Scalability: Supports GPU-accelerated workloads, making it ideal for high-performance AI applications.
• Remote Attestation: Provides cryptographic proofs verifying the integrity and authenticity of the application’s execution.

Introducing Exponent AI’s Vector3 Network

Exponent AI’s Vector3 is a decentralized infrastructure designed as the natural language layer for Web3. It offers an ecosystem of open-source language models capable of answering complex queries using Retrieval-Augmented Generation (RAG), trained on data generated by a network of LLM nodes.

The goal of the Vector3 network is to grow inference datasets by leveraging token rewards to the winning LLM inference. Vector3 will implement a cutting edge research in AI known as LLM as a Judge in a trustless fashion. This AI model judges the output of other LLMs with an objective score, acting as a verifier in the system. Therefore, there is a real need to run these more powerful AI models and ensure that these scorer node runners execute with verifiably correct model, prompt parameters and avoid collusion and biases in the system

Highlights of Vector3:

• Trustless Participation: Incentivizes participants to participate in contribution of query and answers, and subsequently verify data via a Scorer node
• Scorer Nodes: utilizing a more powerful state of the art model, they evaluate model outputs against predefined parameters, ensuring quality and accountability within the system that rewards the best results objectively. Each node runs a ‘Verifier AI’ model.

Why Marlin Oyster and Vector3 are a Perfect Match

Verifiable AI at Scale

With Marlin Oyster’s robust TEE capabilities, Vector3 can execute large-scale verifiable inference tasks securely and efficiently. By isolating the inference process, Oyster guarantees that results are genuine and tamper-proof.

Enhanced Model Trust

Vector3’s reliance on scorer nodes aligns seamlessly with Oyster’s remote attestation. This integration ensures that the outputs of models like GPT-4o or Claude are validated in a trustless environment, addressing the challenges of LLM-as-a-judge use cases.

Confidential Data Handling

Oyster’s confidentiality guarantees protect sensitive datasets used in Vector3’s training and inference pipelines. This safeguards both user queries and proprietary datasets, fostering greater adoption and trust.

Optimized Performance

Oyster’s support for GPU acceleration enables efficient execution of computationally intensive AI tasks, including the ability to run large open source models as a judge, such as Llama 400B parameters

How TEE-Based Verifier AI Works

TEE-based verifier AI leverages trusted execution environments to ensure secure and transparent computations. Here’s an outline of the process:

• A centralized sequencer publishes a query and its corresponding answer on-chain with a unique ID.
• The Marlin Oyster serverless function receives a ping to verify the query and answer.
• It computes the verification, generates a score for the answer, and writes the score on-chain along with a remote attestation proof of computation.

This mechanism ensures that scorer nodes maintain high integrity and prevent collusion or biases by relying on attested execution environments for scoring operation.

The Bigger Picture

This partnership underscores the potential of combining secure computation with decentralized AI. By integrating Marlin Oyster’s TEE platform with Vector3’s decentralized infrastructure, Exponent AI is not only setting a new standard for verifiable 'verifier AI’ but also unlocking novel use cases for TEE that demonstrates real demands for AI security, scalability, and reproducibility.

Together, Exponent AI and Marlin are paving the way for a future where AI and Web3 seamlessly converge, empowering developers and users alike to build and trust the next generation of decentralized LLM compute.

References:

LLM as a Judge for LLM verifier: https://arxiv.org/abs/2411.15594

Learn more about Vector3: https://exponent-cx.notion.site/vector3-litepaper

Learn about Marlin’s Verifiable Compute: https://docs.marlin.org/oyster/introduction-to-marlin/verifiable-computing

Stay Connected

Learn more about Marlin Oyster at Marlin’s Website.
Explore Vector3’s capabilities at Exponent AI.

About Exponent

Exponent is building a natural language layer of web3, revolutionizing information networks for AI and Web3 natives. It has successfully launched Analyst.chat, a fully developed AI agent chat interface platform that takes web3 questions and provides data-backed research and analysis reports in real time. Exponent recently introduced Vector3, an open family of decentralized LLMs and RAG network for Web3 AI Agents. 

Exponent’s mission is to be the leading infrastructure developer that advances the flow of information between Artificial Intelligence and Web3, providing increased access and enhancing capabilities to both fields by using advanced and novel implementations of Artificial Intelligence and Decentralized Infrastructure. 

Website: https://www.exponent.ai/ 
Documentation: https://docs.exponent.ai/ 
Socials: https://qrco.de/exponent_ai
Blog: https://blog.exponent.ai

About Marlin

Marlin is a distributed compute network which uses Trusted Execution Environments (TEEs) to ensure the correctness of computed results. The hardware-based isolation guarantees provided by TEEs allow both short bursts of serverless jobs as well as persistent backend services to be executed with server-grade efficiency, while maintaining the confidentiality of user data. As a privacy-preserving coprocessor for L1s and L2s alike, Marlin drives innovation in DeFi, AI and DeSci.

Website: https://www.marlin.org  
X: https://x.com/MarlinProtocol  
Docs: docs.marlin.org