Overview

ORA provides chain-agnostic infrastructure that bridges the gap between AI and blockchain.

We empower developers with the tools necessary to build end-to-end trustless and decentralized applications enhanced by verifiable AI.

ORA is live today, offering the capability to verify and run inference on the largest and most sophisticated AI models with few limitations.

Our Mission

At ORA, we believe the intersection of AI and crypto unlocks entirely new use cases.

Our mission is to provide developers and organizations with accessible, scalable, and verifiable AI solutions that seamlessly integrate on-chain. By providing these tools and enabling the tokenization of AI models, we create a unified platform that not only advances decentralized AI applications but also funds open-source AI development.

Our Offerings

AI Oracle

AI Oracle is ORA's verifiable and decentralized AI oracle, enabling developers to integrate AI functionalities into their smart contracts seamlessly. The AI Oracle is supported by ORA's network of permissionless nodes running the TORA client and is secured by optimistic Machine Learning (opML).

• Verifiable AI Inference: Bring AI computations onchain with assurance of correctness and trustlessness.

• Developer-Friendly Integration: Build custom smart contracts that interact with the AI Oracle to utilize AI models.

• Use Cases: From AI-generated NFTs and onchain games to prediction markets and content verification, ORA provides the tools necessary to build your unique application.

Initial Model Offerings (IMO)

IMOs represent a new mechanism for tokenizing AI models onchain, promoting sustainable funding for open-source contributors while distributing revenue to tokenholders.

• Tokenization of AI Models: Share ownership and profits of AI models through ERC-7641.

• Sustainable Open-Source Funding: Encourage ongoing contributions and development in the open-source AI community.

• Benefits to Tokenholders: Capture value from onchain revenue and AI-generated content assets.

ORA was previously named HyperOracle.

Trustless AI Oracles for Smart Contracts

• Verifiable AI Computations: Use ORA's AI Oracle to obtain AI inference results that are verified onchain, eliminating the need to trust external APIs.

• Enhanced Decision Making: Implement AI-driven logic in smart contracts for applications like automated trading strategies, dynamic NFTs, or adaptive DeFi protocols.

AI-Generated Content (AIGC) and NFTs

• Verifiable AI Art and Media: Create and tokenize AI-generated art, music, or writing with guaranteed provenance using models like Stable Diffusion.

• ERC7007 Tokens: Mint verifiable AI-generated content as NFTs, providing transparency and trust in the origin of digital collectibles.

• Interactive Experiences: Develop games and virtual environments where AI generates dynamic content, enhancing user engagement with verifiable AI elements.

Sustainable Funding for Open-Source AI

• Initial Model Offerings (IMOs): Tokenize AI models using the ERC7641 standard to fund open-source AI development sustainably.

• Community Ownership: Align incentives between developers and stakeholders by allowing token holders to share in the revenue generated by AI models.

• Accelerated Innovation: Encourage continuous contributions to open-source AI projects by providing a financial ecosystem that rewards development.

Improved Protocol Efficiency and Autonomy

• Automated Governance: Utilize AI models to analyze proposals and assist in decision-making processes within DAOs.

• Risk Management: Implement AI for real-time risk assessment in lending platforms or asset management protocols.

• Optimized Yield Strategies: Use AI to enhance yield farming and liquidity provision, maximizing returns for DeFi users.

Advanced Financial Instruments

• Prediction Markets: Employ AI for accurate forecasting and automated settlement, improving efficiency and trust in prediction markets.

• Underwriting and Credit Scoring: Use verifiable AI models to assess borrower credibility, enabling undercollateralized lending solutions.

• Onchain Insurance: Develop insurance products with AI-driven claim verification and risk assessment, streamlining the insurance process.

Enhanced User Experience and Accessibility

• AI-Powered Wallets: Integrate AI for improved security features like fraud detection and intent recognition in crypto wallets.

• Personalized Interactions: Offer users tailored experiences in decentralized applications through AI-driven personalization and recommendations.

• Language and Accessibility Tools: Utilize AI for translation and accessibility, making blockchain technologies more inclusive globally.

AI Agents and Autonomous Systems

• Agentic Architectures: Create verifiable AI agents that can own assets, execute transactions, and interact autonomously with blockchain applications.

• Verification Services for AI Agents: Ensure that AI agents operate correctly and securely through ORA's verification mechanisms.

• New Economic Models: Explore innovative models where AI agents participate in economies, such as automated trading bots or virtual assistants.

The ORA team has been tirelessly publishing research and shipping developer tools, read about our key milestones here:

• Introduced opML: A novel fraud-proof mechanism enabling the verification of large AI models directly on-chain. Read the research paper.

• Invented opp/ai: The first combined opML and zkML framework, offering both the scalability of opML and the guarantees of zkML. Read the research paper.

• Pioneered Verified Inference for Stable Diffusion: Achieved verified inference for Stable Diffusion on-chain, paving the way for bringing verifiable AI-generated content into the blockchain ecosystem.

• Integrated the First Large Language Model (LLM) Onchain: Successfully brought LlaMA2-7B outputs on-chain, making advanced AI models accessible within blockchain applications.

• Launched the First AI Model Tokenization: Introduced tokenizing AI models to fund open-source development, enabling the first tokenized model $OLM based on OpenLM through Initial Model Offerings (IMOs).

In this tutorial, we’ll explore advanced techniques for interacting with the AI Oracle. Specifically, we’ll dive into topics like Nested Inference, Batch Inference, and Data Availability (DA) Options, giving you a deeper understanding of these features and how to leverage them effectively.

1. Nested Inference

A user can perform nested inference by initiating a second inference based on the result of the first inference within a smart contract. This action can be completed atomically and is not restricted to a two-step function.

Nested Inference Use Cases

Some of the use cases for a nested inference call include:

• generating a prompt with LLM for AIGC (AI Generated Content) NFT

• extracting data from a data set, then generate visual data with different models

• adding transcript to a video, then translate it to different languages with different models

For demo purposes we built a farcaster frame that uses ORA's AI Oracle.

Implementing Nested Inference

The idea of Nested Inference contract is to execute multiple inference requests in 1 transaction. We'll modify Prompt contract to support nested inference request. In our example, it will call Llama3 model first, then use inference result as the prompt to another request to StableDiffusion model.

The main goal of this tutorial is to understand what changes we need to make to Prompt contract in order to implement logic for various use cases.

Implementation Steps

1. modify CalculateAIResult method to support multiple requests

2. modify aiOracleCallback with the logic to handle second inference request

💡 When estimating gas cost for the callback, we should take both models into the consideration.

CalculateAIResult

As we now have additional function parameter for second model id. Not that we encode and forward model2Id as a callback data in aiOracle.requestCallback call.

 function calculateAIResult(uint256 model1Id, uint256 model2Id, string calldata model1Prompt) payable external returns (uint256) {
    bytes memory input = bytes(model1Prompt);
    uint256 model1Fee = estimateFee(model1Id);
    uint256 requestId = aiOracle.requestCallback{value: model1Fee}(
        model1Id, input, address(this), callbackGasLimit[model1Id], abi.encode(model2Id)
    );
    AIOracleRequest storage request = requests[requestId];
    request.input = input;
    request.sender = msg.sender;
    request.modelId = model1Id;
    emit promptRequest(requestId, msg.sender, model1Id, model1Prompt);
    return requestId;
}

aiOracleCallback

The main change here is the within "if" block. If the callback data (model2Id) is returned, we want to execute second inference request to the AI Oracle.

Output from the first inference call, will be passed to second one. This allows for interesting use cases, where you can combine text-to-text (eg. Llama3) and text-to-image (eg. Stable-Diffusion) models.

If nested inference call is not successful the whole function will revert.

💡 When interacting with the contract from the client side, we need to pass cumulative fee (for both models), then for each inference call we need to pass part of that cumulative fee. This is why we are calling estimateFee for model2Id.

function aiOracleCallback(uint256 requestId, bytes calldata output, bytes calldata callbackData) external payable override onlyAIOracleCallback() {
    AIOracleRequest storage request = requests[requestId];
    require(request.sender != address(0), "request does not exist");
    request.output = output;
    prompts[request.modelId][string(request.input)] = string(output);

    //if callbackData is not empty decode it and call another inference
    if(callbackData.length != 0){
        (uint256 model2Id) = abi.decode(callbackData, (uint256));
        uint256 model2Fee = estimateFee(model2Id);

        (bool success, bytes memory data) = address(aiOracle).call{value: model2Fee}(abi.encodeWithSignature("requestCallback(uint256,bytes,address,uint64,bytes)", model2Id, output, address(this), callbackGasLimit[model2Id], ""));
        require(success, "failed to call nested inference");

        (uint256 rid) = abi.decode(data, (uint256));
        AIOracleRequest storage recursiveRequest = requests[rid];
        recursiveRequest.input = output;
        recursiveRequest.sender = msg.sender;
        recursiveRequest.modelId = model2Id;
        emit promptRequest(rid, msg.sender, model2Id, "");
    }

    emit promptsUpdated(requestId, request.modelId, string(request.input), string(output), callbackData);
}

Interaction with Contract

This is an example of contract interaction from Foundry testing environment. Note that we're estimating fee for both models and passing cumulative amount during the function call (we're passing slightly more to ensure that the call will execute if the gas price changes).

PromptNestedInference prompt = new PromptNestedInference(IAIOracle(OAO_PROXY));

uint256 stableDiffusionFee = prompt.estimateFee(STABLE_DIFFUSION_ID);
uint256 llamaFee = prompt.estimateFee(LLAMA_ID);

uint256 requestId = prompt.calculateAIResult{value: ((stableDiffusionFee + llamaFee)*11/10)}(STABLE_DIFFUSION_ID, LLAMA_ID, SD_PROMPT);

Conclusion

You can also check the full implementation of PromptNestedInference.

2. Batch Inference

The Batch Inference feature enables sending multiple inference requests within a single transaction, reducing costs by saving on network fees and improving the user experience. This bulk processing allows for more efficient handling of requests and results, making it easier to manage the state of multiple queries simultaneously.

Some of the use cases might be:

• AIGC NFT marketplace - creating a whole AIGC NFT collection with just one transaction, instead of creating those with many transactions

• Apps that need to handle requests simultaneously - Good example would be a recommendation system or chatbot with high TPS

Pricing

Model fee required for batch inference request is calculated by multiplying single model fee with batch size (batchSize * model.fee). This fee covers the operational costs of running AI models, with a portion contributing to protocol revenue. For details on the required fees for each model, visit the References page.

Callback transaction fee is needed for AI Oracle to submit callback transaction. It's calculated by multiplying current gas price with callback gas limit for invoked model (gasPrice * callbackGasLimit).

Request transaction fee is regular blockchain fee needed to request inference by invoking aiOracle.requestCallback method.

Total fee is calculated as sum of Model fee, Callback transaction fee and Request transaction fee.

Implementing Batch Inference

In order to initiate batch inference request, we will interact with requestBatchInference method. This method takes additional batchSize parameter, which specifies the amount of requests to the AI Oracle.

function calculateAIResult(uint256 modelId, string calldata prompt, uint256 batchSize) payable external {
    bytes memory input = bytes(prompt);
    bytes memory callbackData = bytes("");
    address callbackAddress = address(this);
    uint256 requestId = aiOracle.requestBatchInference{value: msg.value}(
        batchSize, modelId, input, callbackAddress, callbackGasLimit[modelId], callbackData, IAIOracle.DA(0), IAIOracle.DA(0)
    );
}

Note that we'll need to pass more gas to cover AI Oracle callback execution, depending on the batchSize (check out Pricing). For this purpose we can implement estimateFeeBatch function in our Prompt contract. This method will interact with estimateFeeBatch method from AIOracle.sol.

//Prompt.sol
function estimateFeeBatch(uint256 modelId) public view returns (uint256) {
    return aiOracle.estimateFee(modelId, callbackGasLimit[modelId]);
}

//AIOracle.sol
function estimateFeeBatch(uint256 modelId, uint256 gasLimit, uint256 batchSize) public view ifModelExists(modelId) returns (uint256) {
    ModelData storage model = models[modelId];
    return batchSize * model.fee + gasPrice * gasLimit;
}

Prompt Format

Input for batch inference should be structured as a string representing an array of prompt and seed values.

Prompt - string value that is mandatory in order to prompt AI Oracle.

Seed – an optional numeric value that, when used, allows you to generate slightly varied responses for the same prompt.

[
    // Batch inference requests with required "prompt" and optional "seed" or "seed_range".
    {
        "prompt": "prompt1",  // Mandatory prompt
        "seed": 1             // Optional seed for response variation
    },
    {
        "prompt": "prompt2",
        "seed": 2
    },
    {
        "prompt": "prompt3"    // No seed provided, default behavior
    },
    {
        "prompt": "prompt4",
        "seed_range": [1, 2]   // Seed range for controlled variation
    }
]

Output Format

Result of Batch inference call is a dot separated list of inference results.

result1.result2.result3...

Example

This is the prompt for interacting with Stable Diffusion model:

[{"prompt":"generate image of an elephant","seed":1},{"prompt":"generate image of an elephant","seed":2}]

Result is dot separated list of ipfs CIDs:

.Qma3gR6z9dtSQ3fsoSUwQtHnNMYZEHhuJx3DBqx677HPaj.QmemPcnzdyigx1XnAvskVKmEhnsivBuE3Qkp3jJqfdWWxC

Interaction with Batch Inference

When performing batch inference with the AI Oracle, ensure the prompt is following the standard format. Below is a simple script for interacting with batch inference:

import { Web3 } from "web3";

const web3 = new Web3(process.env.RPC_URL);

const wallet = web3.eth.accounts.wallet.add(process.env.PRIVATE_KEY); // Make sure you have funds

const contract = new web3.eth.Contract(batchInference_abi, batchInference_address);

const prompt = `[{"prompt":"what's the best fruit", "seed":1},{"prompt":"what's the best fruit", "seed":2},{"prompt":"kiwi"},{"prompt":"dog", "seed_range":[1,2]}]`

const fee = Number(await contract.methods.estimateFeeBatch(11, 5).call());

const totalFee = (fee*11/10)

const tx = await contract.methods.requestBatchInference(11, prompt, 5).send({from: wallet[0].address, value: totalFee});
console.log("Tx: ", tx)

setTimeout(async () => {
    const result = await contract.methods.prompts(11, prompt).call();
    console.log("Result: ", result)
}, 30000);

To test it, you need to:

1. create new javascript file

2. copy the script and add env variables

3. create and deploy prompt contract that supports batch inference

4. add values to batchInference_abi and batchInference_address

Prompt examples

// Example 1: Batch with identical prompts but different seeds for varied responses
const prompt1 = `[{"prompt":"hello","seed":1},{"prompt":"hello","seed":2}]`

// Example 2: Batch with different prompts, no seeds specified, uses default behavior
const prompt2 = `[{"prompt":"hello"},{"prompt":"hello world"}]`

// Example 3: Batch with same question but varied seeds, and different prompts with a seed range
const prompt3 = `[{"prompt":"what's the best fruit", "seed":1},{"prompt":"what's the best fruit", "seed":2},{"prompt":"kiwi"},{"prompt":"dog","seed_range":[1,2]}]`

// Example 4: Batch with identical prompt but different seeds, plus a seed range for a specific prompt
const prompt4 = `[{"prompt":"apple", "seed":1}, {"prompt":"apple", "seed":2}, {"prompt":"banana"}, {"prompt":"cat", "seed_range": [1,100] }]`

Conclusion

That's it! With few simple changes to the Prompt contract we utilised batch inference feature. This allowed us to get multiple responses with only one transaction.

ORA's AI Oracle is designed with decentralization at its core. It ensures that AI inference results are verifiable onchain using opML and censorship-resistant. ORA's network is built on a permissionless node structure, allowing anyone to run the node client software.

When interacting with the AI Oracle, it is crucial to allocate sufficient gas for the execution of the callback function. The gas consumed during the callback depends entirely on the implementation of the aiOracleCallback method. Its logic directly impacts gas usage - the more complex the logic, the higher the gas consumption. Carefully design your aiOracleCallback to balance functionality and gas efficiency.

To estimate the gas required for the aiOracleCallback in a Foundry project:

1. Run the following command to generate a gas report:

   Copy

   forge snapshot --gas-report

2. Locate your Prompt contract in the report and check the gas usage for the aiOracleCallback method.

3. Use the calculated gas amount to set the gas limit during the initialisation of the Prompt contract (in the constructor).

Alternatively you can use other tools like Hardhat or Tenderly.

Test Script

• We provided test script for gas limit estimation in the template repository.

• To execute estimation run: forge test test/EstimateGasLimit.t.sol -vvvv . Next to the callback function, you can observe the gas amount needed for execution. eg. [1657555] Prompt::aiOracleCallback -> in this case amount we can set gas limit to 1700000

// SPDX-License-Identifier: UNLICENSED
pragma solidity ^0.8.13;

import {Test, console2, Vm} from "forge-std/Test.sol";
import {Prompt} from "../src/Prompt.sol";
import {IAIOracle} from "OAO/contracts/interfaces/IAIOracle.sol";
import {OraSepoliaAddresses} from "./OraSepoliaAddresses.t.sol";
import "forge-std/console.sol";

contract EstimateGasLimitTest is Test, OraSepoliaAddresses {
    Prompt prompt;
    string rpc;
    uint256 forkId;
    uint256 modelId;
    string result;

    function setUp() public {
        rpc = vm.envString("RPC_URL");
        forkId = vm.createSelectFork(rpc);
        prompt = new Prompt(IAIOracle(OAO_PROXY));
        modelId = 50;
        result = "Qmd3xWJVRao8AfgRTuXLCWBYj6VdVV8HFuKygk9FLTW5bi";
    }

    function test_estimateGasLimit() public {
        uint256 requestId = prompt.calculateAIResult{value: prompt.estimateFee(modelId)}(modelId, "test generation");
        uint256 before = gasleft();
        vm.prank(OAO_PROXY);
        prompt.aiOracleCallback(requestId, bytes(result), "");
        uint256 afterCall = gasleft();
        console.log(before - afterCall);
    }
}

The above script estimates gas necessary for Stable-Diffusion callback. Note that result will always be constant size (ipfs CID).

Llama3

To estimate gas limit for Llama3, you need to change modelId and result.

modelId = 11;
result = "Beneath the sky, where whispers blend, A story stirs that has no end. The trees bow low, the rivers hum, A timeless tune, where life has sprung. The sun ascends with golden rays, To bless the fields and mark the days. The flowers bloom, their colors speak, Of beauty vast, profound, unique. The stars alight, a cosmic sea, Of dreams untold and destiny. Each twinkle holds a secret dear, A tale of love, of hope, of fear. Through valleys deep and mountains wide, The journey calls, the hearts true guide. With every step, a lesson found, In fleeting moments, wisdoms ground. Yet in the shadows, doubts may creep, A restless thought, a troubled sleep. But courage whispers, soft and clear, Youre not alone; Im always near. So carry forth, embrace the storm, Through fire and frost, the heart is warm. For lifes a dance, both wild and free, A fleeting song of harmony. As time unfurls its endless thread, The soul will soar where dreams are led. A symphony of lifes embrace, endless journey through boundless space. Beneath the moon, the rivers gleam, A quiet whisper shapes a dream. Through winds of change and skies of blue, I love you.";

💡 The maximum length of Llama3 response is 200 words string, hence we will use this string size to estimate gas limit.

This tutorial will help you understand the structure of the AI Oracle, guide you through the process of building a simple Prompt contract that interacts with the ORA network.

If you prefer a video version of the tutorial, check it here.

Final version of the code can be found here.

Learning Objectives

• Setup the development environment

• Understand the project setup and template repository structure

• Learn how to interact with the AI Oracle and build an AI powered smart contract

Prerequisites

To follow this tutorial you need to have Foundry and git installed.

Setup

1. Clone template repository and install submodules

git clone -b OAO_interaction_tutorial git@github.com:ora-io/Interaction_With_OAO_Template.git --recursive

1. Move into the cloned repository

cd Interaction_With_OAO_Template

1. Copy .env.example, rename it to .env. We will need these env variables later for the deployment and testing. You can leave them empty for now.

cp .env.example .env

1. Install foundry dependencies

forge install

Creating Prompt contract

1. Import dependencies

At the beginning we need to import several dependencies which our smart contract will use.

import "OAO/contracts/interfaces/IAIOracle.sol";
import "OAO/contracts/AIOracleCallbackReceiver.sol";

IAIOracle - interface that defines a requestCallback method that needs to be implemented in the Prompt contract AIOracleCallbackReceiver - an abstract contract that contains an instance of AIOracle and implements a callback method that needs to be overridden in the Prompt contract

2. Write the code

We'll start by implementing the constructor, which accepts the address of the deployed AIOracle contract.

constructor(IAIOracle _aiOracle) AIOracleCallbackReceiver(_aiOracle){}

Now let’s define a method that will interact with the AI Oracle. This method requires 2 parameters, id of the model and input prompt data. It also needs to be payable, because a user needs to pass the fee for the callback execution.

function calculateAIResult(uint256 modelId, string calldata prompt) payable external {
    bytes memory input = bytes(prompt);
    bytes memory callbackData = bytes("");
    address callbackAddress = address(this);
    uint256 requestId = aiOracle.requestCallback{value: msg.value}(
        modelId, input, callbackAddress, callbackGasLimit[modelId], callbackData
    );
}

In the code above we do the following:

1. Convert input to bytes

2. Call the requestCallback function with the following parameters:

   • modelId: ID of the AI model in use.

   • input: User-provided prompt.

   • callbackAddress: The address of the contract that will receive AI Oracle's callback.

   • callbackGasLimit[modelId]: Maximum amount of that that can be spent on the callback, yet to be defined.

   • callbackData: Callback data that is used in the callback.

Next step is to define the mapping that keeps track of the callback gas limit for each model and set the initial values inside the constructor. We’ll also define a modifier so that only the contract owner can change these values.

address owner;

modifier onlyOwner() {
    require(msg.sender == owner, "Only owner");
    _;
}

mapping(uint256 => uint64) public callbackGasLimit;

function setCallbackGasLimit(uint256 modelId, uint64 gasLimit) external onlyOwner {
    callbackGasLimit[modelId] = gasLimit;
}

constructor(IAIOracle _aiOracle) AIOracleCallbackReceiver(_aiOracle) {
    owner = msg.sender;
    callbackGasLimit[50] = 500_000; // Stable-Diffusion
    callbackGasLimit[11] = 5_000_000; // Llama
}

💡 Gas Limit Estimation - It's important to define callbackGasLimit for each model to ensure that AIOracle will have enough funds to return the response! Read more on Callback gas limit Estimation page.

We want to store all the requests that occurred. For that we can create a data structure for the request data and a mapping between requestId and the request data.

event promptRequest(
    uint256 requestId,
    address sender, 
    uint256 modelId,
    string prompt
);

struct AIOracleRequest {
    address sender;
    uint256 modelId;
    bytes input;
    bytes output;
}

mapping(uint256 => AIOracleRequest) public requests;

function calculateAIResult(uint256 modelId, string calldata prompt) payable external {
    bytes memory input = bytes(prompt);
    bytes memory callbackData = bytes("");
    address callbackAddress = address(this);
    uint256 requestId = aiOracle.requestCallback{value: msg.value}(
        modelId, input, callbackAddress, callbackGasLimit[modelId], callbackData
    );
    AIOracleRequest storage request = requests[requestId];
    request.input = input;
    request.sender = msg.sender;
    request.modelId = modelId;
    emit promptRequest(requestId, msg.sender, modelId, prompt);
}

In the code snippet above we added prompt, sender and the modelId to the request and also emitted an event.

Now that we implemented a method for interaction with the AI Oracle, let's define a callback that will be invoked by the AI Oracle after the computation of the result.

event promptsUpdated(
    uint256 requestId,
    uint256 modelId,
    string input,
    string output,
    bytes callbackData
);

mapping(uint256 => mapping(string => string)) public prompts;

function getAIResult(uint256 modelId, string calldata prompt) external view returns (string memory) {
    return prompts[modelId][prompt];
}

function aiOracleCallback(uint256 requestId, bytes calldata output, bytes calldata callbackData) external override onlyAIOracleCallback() {
    AIOracleRequest storage request = requests[requestId];
    require(request.sender != address(0), "request does not exist");
    request.output = output;
    prompts[request.modelId][string(request.input)] = string(output);
    emit promptsUpdated(requestId, request.modelId, string(request.input), string(output), callbackData);
}

We've overridden the callback function from the AIOracleCallbackReceiver.sol. It is important to use the modifier, so that only AI Oracle can callback into our contract.

Function flow:

1. First we check if the request with provided id exists. If it does we add the output value to the request.

2. Then we define prompts mapping that stores all the prompts and outputs for each model that we use.

3. At the end we emit an event that the prompt has been updated.

Notice that this function takes callbackData as the last parameter. This parameter can be used to execute arbitrary logic during the callback. It is passed during requestCallback call. In our simple example, we left it empty. Finally let's add the method that will estimate the fee for the callback call.

function estimateFee(uint256 modelId) public view returns (uint256) {
    return aiOracle.estimateFee(modelId, callbackGasLimit[modelId]);
}

With this, we've completed with the source code for our contract. The final version should look like this:

// SPDX-License-Identifier: MIT
pragma solidity ^0.8.13;

import "OAO/contracts/interfaces/IAIOracle.sol";
import "OAO/contracts/AIOracleCallbackReceiver.sol";

contract Prompt is AIOracleCallbackReceiver {
    
    event promptsUpdated(
        uint256 requestId,
        uint256 modelId,
        string input,
        string output,
        bytes callbackData
    );

    event promptRequest(
        uint256 requestId,
        address sender, 
        uint256 modelId,
        string prompt
    );

    struct AIOracleRequest {
        address sender;
        uint256 modelId;
        bytes input;
        bytes output;
    }

    address owner;

    modifier onlyOwner() {
        require(msg.sender == owner, "Only owner");
        _;
    }

    mapping(uint256 => AIOracleRequest) public requests;

    mapping(uint256 => uint64) public callbackGasLimit;

    constructor(IAIOracle _aiOracle) AIOracleCallbackReceiver(_aiOracle) {
        owner = msg.sender;
        callbackGasLimit[50] = 500_000; // Stable-Diffusion
        callbackGasLimit[11] = 5_000_000; // Llama
    }

    function setCallbackGasLimit(uint256 modelId, uint64 gasLimit) external onlyOwner {
        callbackGasLimit[modelId] = gasLimit;
    }

    mapping(uint256 => mapping(string => string)) public prompts;

    function getAIResult(uint256 modelId, string calldata prompt) external view returns (string memory) {
        return prompts[modelId][prompt];
    }

    function aiOracleCallback(uint256 requestId, bytes calldata output, bytes calldata callbackData) external override onlyAIOracleCallback() {
        AIOracleRequest storage request = requests[requestId];
        require(request.sender != address(0), "request does not exist");
        request.output = output;
        prompts[request.modelId][string(request.input)] = string(output);
        emit promptsUpdated(requestId, request.modelId, string(request.input), string(output), callbackData);
    }

    function estimateFee(uint256 modelId) public view returns (uint256) {
        return aiOracle.estimateFee(modelId, callbackGasLimit[modelId]);
    }

    function calculateAIResult(uint256 modelId, string calldata prompt) payable external {
        bytes memory input = bytes(prompt);
        bytes memory callbackData = bytes("");
        address callbackAddress = address(this);
        uint256 requestId = aiOracle.requestCallback{value: msg.value}(
            modelId, input, callbackAddress, callbackGasLimit[modelId], callbackData
        );
        AIOracleRequest storage request = requests[requestId];
        request.input = input;
        request.sender = msg.sender;
        request.modelId = modelId;
        emit promptRequest(requestId, msg.sender, modelId, prompt);
    }
}

Deployment and interaction with the contract

Foundry version

1. Add your PRIVATE_KEY, RPC_URL and ETHERSCAN_KEY to .env file. Then source variables in the terminal.

   Copy

   source .env

2. Create a deployment script

   Go to Reference page and find the OAO_PROXY address for the network you want to deploy to.

   Then open script/Prompt.s.sol and add the following code:

   Copy

   // SPDX-License-Identifier: MIT
   pragma solidity ^0.8.13;
   
   import {Script} from "forge-std/Script.sol";
   import {Prompt} from "../src/Prompt.sol";
   import {IAIOracle} from "OAO/contracts/interfaces/IAIOracle.sol";
   
   contract PromptScript is Script {
       address OAO_PROXY;
   
       function setUp() public {
           OAO_PROXY = [OAO_PROXY_address_here];
       }
   
       function run() public {
           uint privateKey = vm.envUint("PRIVATE_KEY");
           vm.startBroadcast(privateKey);
           new Prompt(IAIOracle(OAO_PROXY));
           vm.stopBroadcast();
       }
   }

3. Run the deployment script

   Copy

   forge script script/Prompt.s.sol --rpc-url $RPC_URL --broadcast --verify --etherscan-api-key $ETHERSCAN_KEY

4. Once the contract is deployed and verified, you can interact with it. Go to blockchain explorer for your chosen network (eg. Etherscan), and paste the address of the deployed Prompt contract.

   Let's use Stable Diffusion model (id = 50) as an example.

   1. First call estimateFee method to calculate fee for the callback.
   2. Request the AI inference from the AI Oracle by calling calculateAIResult method. Pass the model id and the prompt for the image generation. Remember to provide an estimated fee as a value for the transaction.
   3. After the transaction is executed, and the AI Oracle calculates the result, you can check it by calling prompts method. Simply input model id and the prompt you used for image generation. In the case of Stable Diffusion, the output will be a CID (content identifier on ipfs). To check the image go to https://ipfs.io/ipfs/[Replace_your_CID].

Remix version

1. Install the browser wallet if you haven't already (eg. Metamask)

2. Open your solidity development environment. We'll use Remix IDE.

3. Copy the contract along with necessary dependencies to Remix.
4. Choose the solidity compiler version and compile the contract to the bytecode
5. Deploy the compiled bytecode Once we compiled our contract, we can deploy it.

   1. First go to the wallet and choose the blockchain network for the deployment.

   2. To deploy the Prompt contract we need to provide the address of already deployed AIOracle contract. You can find this address on the reference page. We are looking for OAO_PROXY address.

   3. Deploy the contract by signing the transaction in the wallet.
6. Once the contract is deployed, you can interact with it. Remix supports API for interaction, but you can also use blockchain explorers like Etherscan. Let's use Stable Diffusion model (id = 50).

Conclusion

In this tutorial we covered a step by step writing of a solidity smart contract that interacts with ORA's AI Oracle. Then we compiled and deployed our contract to the live network and interacted with it. In the next tutorial, we will extend the functionality of the Prompt contract to support AI generated NFT collections.

Resources

Source code of AI Oracle: https://github.com/ora-io/OAO

For supported models and deployment addresses, see Reference page.

For example integrations and ideas to build, see awesome-ora.

Check out video tutorial on interacting and building with AI oracle.

ORA's AI Oracle is a verifiable oracle protocol that allows developers to create smart contracts and applications that ingest verifiable machine learning (ML) inferences for uses on the blockchain.

AI Oracle is powered by Optimistic Machine Learning (opML). It enables a verifiable, transparent, and decentralized way to integrate advanced ML models like LLaMA 3 and Stable Diffusion into smart contracts.

It offers:

• Scalability: Can run any ML model onchain without prohibitive overhead.

• Efficiency: Reduces computational time and cost when compared to zkML.

• Practicality: Can easily be integrated into applications by using existing blockchain infrastructure.

• Verifiability: Leverages fraud proofs to provide applications and users with assured computational integrity.

Components

AI Oracle comprises a set of smart contracts and off-chain components:

1. opML Contract: Handles fraud proofs and challenges to ensure on-chain verifiability of ML computations.

2. AIOracle Contract: Connects the opML node with on-chain callers to process ML requests and integrates various ML models.

3. User Contract: Customizable contracts that initiate AI inference requests and receive results from directly from AI Oracle.

4. ORA Nodes: Machines that run the Tora Client that interact with the ORA network. Nodes currently perform two functions: submit or validate inference results.

We equip developers with the essential tools to create decentralized, trustless applications powered by verifiable AI.

Introduction

Unveiled by the ORA Foundation, $ORA is the token of ORA ecosystem, designed to drive advancements in blockchain intelligence through decentralized AI.

$ORA's utility includes access to IMO, node operations, governance, and more.

About $ORA

$ORA Addresses

Markets

$ORA is live and trading at:

• Uniswap v3 Pool on Ethereum Mainnet: 0x4e4a4c4c46d3488ff35ff05a0233785a30f03ec4

• Uniswap v3 Pool on Base Mainnet: 0x316f12517630903035a0e0b4d6e617593ee432ba

• ORA/USDC Pool on Hyperliquid: 0xd7a5b9b760fd758d2e3f6f3ecd2ae5bb

• ORA/SOL Pool on Raydium: BPYhCMNao2XG4UR771LurHrwUg3rcp76jmg4XFfAacvg

For more markets, including centralized exchanges, where you can access $ORA, visit $ORA's CoinMarketCap and $ORA's CoinGecko pages.

Bridge $ORA

On EVM networks (Ethereum, Base...) and Solana, $ORA is OFT token based on LayerZero's OFT standard.

To bridge $ORA token, you can choose to use:

Currently, bridging of $ORA between Hyperliquid is not supported.

Token Statistics

Inspired by Vitalik Buterin’s vision of DAICO (a hybrid of DAO governance for transparency and ICO’s open fundraising), $ORA was launched with Decentralized AI Community Offering.

• Total Supply: 333,333,333

• Circulating Supply: 36,666,666.63

• Token Distribution:

Token Utility

$ORA token is central to the ORA ecosystem, serving multiple roles including:

1. Access to IMO: $ORA grants ecosystem members the access to IMOs, funding the next state-of-the-art AI models and driving innovation in open-source AI.

2. Node Operator: By staking $ORA, operator can run decentralized nodes, supporting services like OAO and RMS while earning fees.

3. Governance: $ORA holders play a crucial role in shaping the future of the protocol through decentralized governance.

4. Others to be announced.

Read More

Learn more about $ORA and the ORA ecosystem:

• ORA Foundation: https://foundation.ora.io

• ORA Foundation's X: https://x.com/FoundationORA

Introduction

The Optimistic Machine Learning (OPML) framework connects off-chain machine learning computations with smart contracts. Results are provably correct and can be disputed on-chain to ensure trust and accountability, enabling efficient AI-driven decision-making in decentralized applications. OPML is a core component of the AI Oracle, playing a critical role in validating inference results.

Workflow

1. Initiate Request: The user contract sends an inference request to AI Oracle by calling the requestCallback function.

2. opML Request: AI Oracle creates an opML request based on the user contract request.

3. Event Emission: AI Oracle emits a requestCallback event collected by the opML node.

4. ML Inference: The opML node performs the AI model computation.

5. Result Submission: The opML node uploads the result on-chain.

6. Callback Execution: The result is dispatched to the user's smart contract via a callback function.

Challenge Process

1. Challenge Window: Begins after the result is submitted on-chain (step 5 above).

2. Verification: opML validators or any network participant can check the results and challenge the output if it is incorrect.

3. Result Update: If a challenge is successful, the incorrect result is updated on-chain.

4. Finality: After the challenge period, the result is finalized onchain and made immutable.

Using AI Oracle

As a user, to interact with AI Oracle, you can:

• Use the AI.ORA.IO frontend- view our video tutorial

• Directly via the Prompt contract on Etherscan

Via Etherscan

1. In Prompt contract's Read Contract section, call estimateFee with specified modelId.

1. In Prompt contract's Write Contract section, call calculateAIResult with fee (converted from wei to ether), prompt, and modelId.

1. In the AI Oracle contract, look for the new transaction that fulfills the request you sent.

1. In Prompt contract's Read Contract section, call getAIResult with previous modelId and prompt to view the AI inference result.

Retrieve Historical AI Inference

If you want to retrieve your historical AI inference (eg. AIGC image), you can find them on blockchain explorer:

1. Find your transaction for sending AI request, and enter the "Logs" tab. Example tx: https://etherscan.io/tx/0xfbfdb2efcee23197c5ea8487368a905385c84afdc465cab43bc1ad01da773404#eventlog

2. Access your requestId in "Logs" tab Example's case: "1928".

3. In OAO's smart contract, look for the Invoke Callback transaction with the same requestId. Normally, this transaction will be around the same time as the one in step 1. To filter transactions by date, click "Advanced Filter" and then the button near "Age".

4. Find the transaction for AI inference result, and enter the "Logs" tab. Example tx: https://etherscan.io/tx/0xfbfdb2efcee23197c5ea8487368a905385c84afdc465cab43bc1ad01da773404#eventlog

5. Access output data. Example's case: "QmecBGR7dD7pRtY48FEKoeLVsmBTLwvdicWRkX9xz2NVvC" which is the IPFS hash that can be accessed with IPFS gateway)

To showcase how AI Oracle can enhance consumer-facing products, we introduce Fortune Teller.

About Fortune Teller

Fortune Teller leverages the NestedInference smart contract, as discussed in the Advanced Usages of AI Oracle. This application aims to onboard new users to Web3 and demonstrate the capabilities of AI Oracle.

Application Flow

1. User Interaction: The application prompts users to answer specific questions posed by the Magic Wizard.

2. Fortune Telling: The Wizard casts its spells by requesting inferences from AI Oracle.

3. Response Generation: The Llama3 model generates a textual response, which is then used as a prompt for image creation via StableDiffusion.

4. NFT Minting: Users can mint AI-generated NFTs if they are satisfied with their fortune image.

The image on the right represents user's fortune result generated by AI Oracle:

• Text: Generated by Llama3 (onchain tx)

• Image: Generated by Stable Diffusion v3 (onchain tx)

Benefits of using AI Oracle in Various Applications

• Objective AI-Generated Insights: AI provides neutral and unbiased outputs, ensuring a fair experience across diverse use cases.

• Immutable Onchain Data: Information generated is securely stored on the blockchain, making it tamper-proof and easily verifiable.

• Transparent Data Generation: Utilizing opML, the entire generation process is transparent, fostering trust in the system across different applications.

Build your own AI dapps!

💡 If you're interested in creating your own frame, check out our AI Oracle Frame Template repository. This template can help you bootstrap your application, but you'll need to modify it based on your specific use case.

We encourage you to develop your own applications. Explore ideas in the awesome-ora repository, whether it’s a Farcaster frame, a Telegram mini app, or any other client application that interacts with AI Oracle.

Introduction

Tora Launcher is a multi-platform Tora client running as a desktop application.

Users with any level of experience can use it to start up and run their own Tora node, just like running a desktop wallet.

Currently, Tora Launcher supports MacOS, Linux, and Windows.

Releases can be found here: Download Tora Launcher.

Guide for MacOS

1) Download

Go to releases page, find the release tagged with "Latest", and download the file ending with .dmg.

2) Install App

Double click the .dmg file you just downloaded, and follow the guide on your device to install.

3) Install Tora Docker Image

After opening up the app, you will need to install the Tora docker image that includes the OpenLM model.

Click on "INSTALL TORA".

It is required to keep docker app open when installing. If you don't have docker open when going into this step, you will need to restart Tora app.

4) Configure Node

After installation, you will see the app's main screen.

Click on "SETTING".

Enter configurations to all required fields:

• PRIVATE KEY: the private key that you use for the node that pays gas with sufficient balance for node operations on blockchain. Please note: it is recommended to use a separate wallet other than your daily used wallet.

• MODEL: the AI model in Onchain AI Oracle that you want to run in your node.

• NETWORK: the network that you submit validation results.

• WEBSOCKET URL and JSON-RPC URL: the RPC endpoints that you use for getting blockchain data. Default RPCs are provided.

Scroll down and click on "START" to start your node.

If you are starting the node for the first time, you are required to set up password.

5) Running the node

When node is running, you will be see the status on the top-right corner and "NODE" page.

You can also check out logs in "LOG" page.

For any question, please join our discord.

Introduction

Tora Validator is a validator client of the AI Oracle network, written in TypeScript.

As the first released version of ORA node, it validates AI inference results submitted by AI Oracle nodes.

To enhance client diversity and further decentralize the network, future updates will introduce additional types of nodes.

Tora Validator Node performs two main functions:

• Validation: It validates AI inference results by executing AI computations within the opML framework.

• Blockchain Interaction: The node interacts with ORA's AI Oracle smart contracts by monitoring AI requests and confirming validated inference results.

Technical Details

Tora Validator supports the OpenLM model (model ID: 13) on the Ethereum mainnet. The node runs this model and confirms its AI inference results on-chain, contributing to the overall security and trustworthiness of the network.

Tutorials

To encourage participation, Tora offers two options of client software for running the node:

• Desktop App (Tora Launcher): A beginner-friendly approach for users who prefer a graphical interface.

• CLI: An advanced option for those comfortable with command-line operations, offering more control and flexibility.

In this tutorial we explain how to integrate your own AI model with ORA's AI Oracle. We will start by looking at repository and trying to understand what's happening there. At the end we will showcase how works, by running a simple dispute game script inside a docker container.

Learning Objectives

• Understand how to transform an AI model inference code and integrate it with ORA's AI Oracle

• Execute a simple dispute game and understand the process of AI inference verification.

Prerequisites

• installed

Setup

1. Clone repository

1. Navigate to cloned repository

1. To install the required dependencies for your project, run the following command:

If there are some missing dependencies, make sure to install them in your Python environment.

Train the model using Pytorch

First we need to train a DNN model using Pytorch. The training part is shown in examples/mnist/trainning/mnist.ipynb.

After the training model is saved at examples/mnist/models/mnist/mnist-small.state_dict.

Convert model into ggml format

1. Position to mnist folder

1. Convert python model into ggml format

In order to convert AI model written in Python to ggml format we are executing python script and providing a file that stores the model as a parameter to the script. The output is the binary file in ggml format. Note that the model is saved in big-endian, making it easy to process in the big-endian MIPS-32 VM.

Converting inference code to MIPS VM executable

Next step is to write inference code in go language. Then we will transform go binary into MIPS VM executable file.

Go supports compilation to MIPS. However, the generated executable is in ELF format. We'd like to get a pure sequence of MIPS instructions instead. To build a ML program in MIPS VM execute the following steps:

1. Navigate to the mnist_mips directory and build go inference code

Running the dispute game

Now that we compiled our AI model and inference code into MIPS VM executable code.

1. First we need to specify the operating system that runs inside our container. In this case we're using ubuntu:22.04.

1. Then we configure ssh keys, so that docker container can clone all the required repositories.

1. Position to the root directory inside docker container and clone opml repository along with its submodules.

1. Lastly, we tell docker to build executables and run the challenge script.

Create docker container and run the script

1. In order to successfully clone opml repository, you need to generate a new ssh key and add it to your Github account. Once it's generated, save the key in the local repository where Dockerfile is placed as id_rsa. Then add the public key to your GitHub account.

2. Build the docker image

3. Run the local Ethereum node

4. In another terminal run the challenge script

After executing the steps above you should be able to see interactive challenge process in the console.

Script first deploys necessary contracts to the local node. Proposer opML node executes AI inference and then the challenger nodes can dispute it, if they think that the result is not valid. Challenger and proposer are interacting in order to find the differing step between their computations. Once the dispute step is found it's sent to the smart contract for the arbitration. If the challenge is successful the proposer node gets slashed.

In this tutorial we achieved the following:

• converted our AI model from Python to ggml format

• compiled AI inference code written in go to MIPS VM executable format

• run the dispute game inside a docker container and understood the opML verification process

General

• Make sure you have downloaded the latest version of the Tora client: https://github.com/ora-io/tora-electron-releases/releases/.

• Make sure you have the official installation of docker.

Issues

Introduction

opp/ai (Optimistic Privacy-Preserving AI on Blockchain) is a novel framework that combines the efficiency of opML (Optimistic Machine Learning) with the privacy guarantees of zkML (Zero-Knowledge Machine Learning). By strategically partitioning machine learning (ML) models, opp/ai balances computational efficiency and data privacy, enabling secure and efficient AI services onchain.

The following is a comprehensive overview of opp/ai, including its architecture, technical components, and use cases.

Key Features of opp/ai

• Privacy Preservation: Protects sensitive data and proprietary models by leveraging Zero-Knowledge Proofs (ZKPs) for critical components.

• Balanced Efficiency: Reduces zkML overhead by limiting its use to essential parts of the model while running the rest in opML.

• Flexibility: Allows customization of the model partitioning to tailor privacy and efficiency based on application requirements.

• Security: Combines cryptographic security from zkML with the economic security model of opML.

How opp/ai Works

Model Partitioning Strategy

opp/ai divides the ML model into submodels based on privacy requirements:

• zkML Submodels:

  • Components that handle sensitive data or proprietary algorithms.

  • Executed using Zero-Knowledge Proofs to ensure data and model confidentiality.

• opML Submodels:

  • Components where efficiency is prioritized over privacy.

  • Executed using the optimistic approach of opML for maximum efficiency.

Example Partitioning:

An ML model M is partitioned into:

• M_zk (zkML Submodel): Handles sensitive computations.

• M_op (opML Submodel): Handles non-sensitive computations.

Hybrid Execution

• Zero-Knowledge Proofs for zkML Submodels:

  • The prover generates ZKPs for M_zk without revealing sensitive data.

  • Proofs are submitted to the blockchain and verified by validators.

• Optimistic Execution for opML Submodels:

  • M_op is executed efficiently off-chain.

  • Results are submitted to the blockchain by decentralized service providers and are assumed correct unless challenged.

• Integration Mechanism:

  • Outputs from M_zk are used as inputs for M_op and vice versa as necessary.

  • Ensures seamless integration of zkML and opML components.

Architecture

opp/ai relies on the three major components of opML for opML submodels:

Fraud Proof Virtual Machine (FPVM)

The FPVM is a specialized virtual machine designed for opML that can:

• Execute and Verify Computation: Capable of performing individual computation steps to resolve disputes.

• State Management: Uses Merkle trees to represent and manage computation states efficiently.

• Onchain Arbitration: Runs on smart contracts, enabling the on-chain verification of disputed steps.

Machine Learning Engine

The ML engine in opML is tailored for both native execution and fraud-proof scenarios:

• Dual Compilation: ML models are compiled twice:

  • Native Execution: Optimized for speed, utilizing multi-threading and GPU acceleration.

  • Fraud-Proof VM Instructions: Compiled into a machine-independent code for use in FPVM during disputes.

• Determinism and Consistency:

  • Fixed-Point Arithmetic: Ensures consistent results across different environments.

  • Software-Based Floating-Point Libraries: Guarantees cross-platform determinism.

Interactive Dispute Game

An efficient protocol for resolving disputes:

• Bisection Protocol:

  • Iteratively narrows down the computation steps to find the exact point of disagreement.

  • Reduces the amount of data and computation needed for onchain verification.

• Onchain Arbitration:

  • Only the minimal necessary computation (usually a single step) is performed onchain.

  • Ensures disputes are resolved fairly and efficiently.

In addition to the opML components above, opp/ai incorporates the following zkML components:

Prover

The prover generates ZKPs, such as zk-SNARKs, to demonstrate the validity of the ML inference.

On-chain Verifier

The verifier smart contract efficiently validates the zero-knowledge proofs provided by the prover.

Workflow

To achieve model privacy, the proofs of all zkML submodels should have their model inputs set as public inputs, while the model parameters or weights should be kept private or hashed. The workflow is as follows:

1. Model Partitioning:

   • The ML model is partitioned into zkML and opML submodels.

   • All submodels running in opML are published as a public model available to all potential submitters and challengers.

2. Request Initiation:

   • A user (requester) submits a request for an ML inference task.

3. Computation and Submission:

   • The prover receives the model input and computes the proof the zkML submodels, submitting them onchain for verifiers while also requesting to initiate opML service tasks for the opML submodels.

   • The submitter performs the opML service task, taking the result from the zkML submodels as input to the opML model and commits the results onchain.

4. Dispute Resolution:

   • The challengers validate the results and will start a dispute process with the submitter if they find any of them inaccurate. The dispute process will only apply to the specific opML submodel being challenged.

   • For opML submodels, disputes are resolved via the interactive dispute game using the FPVM.

   • For zkML submodels, the correctness is ensured by the cryptographic proofs.

5. Finalization:

   • Upon successful verification and resolution of disputes, the final result is confirmed on the blockchain.

Security Considerations

Single Prover Trust Assumption

In the context of preserving model privacy, the single prover trust assumption posits that only one trusted entity, the prover, has access to the model weights. All end users interact with this prover to generate zkML proofs without direct access to the model weights themselves.

This approach is designed to protect the confidentiality of the model while still enabling users to benefit from its predictive capabilities.

Model Extraction Attacks

opp/ai acknowledges that attackers could attempt to reconstruct models through extensive queries due to the single prover trust assumption inherent to zkML.

In order to mitigate this, opp/ai can limit the number of allowed inferences or adjust the cost per inference to deter attacks.

Input Privacy vs. Model Privacy

In order to ensure user input privacy, zkML proof generation should occur on local devices where model weights are distributed. However, this lends the model to reverse engineering attacks.

Given this limitation of zkML, the opp/ai framework instead focuses on model parameter privacy rather than input privacy.

Frequently Asked Questions

1. How does opp/ai differ from opML and zkML individually?

   • opp/ai integrates both opML and zkML to provide a balance between efficiency and privacy. It uses zkML for sensitive computations and opML for the rest, whereas opML focuses on efficiency without privacy, and zkML provides privacy but with higher computational costs.

2. What types of models are suitable for opp/ai?

   • Models that can be partitioned based on privacy requirements. This includes models where certain layers or components handle sensitive data or proprietary algorithms.

3. How do I decide which parts of my model should run in zkML vs. opML?

   • Analyze the model to identify components that require privacy (e.g., layers with proprietary weights or handling sensitive data). These should run in zkML. Components where privacy is not a concern can run in opML for efficiency.

4. What are the hardware requirements for opp/ai?

   • zkML Components: May require machines with substantial resources for proof generation, especially for larger models.

   • opML Components: Can run on standard PCs with CPU/GPU capabilities.

5. Is opp/ai practical for large-scale models?

   • While zkML components can be resource-intensive, opp/ai reduces the overhead by only applying zkML to critical parts of the model. This makes it more practical than using zkML for the entire model.

6. How does opp/ai handle input privacy?

   • Input privacy is challenging due to potential reverse-engineering attacks. opp/ai focuses on model privacy and acknowledges limitations in input privacy with current zkML technologies.

7. Can opp/ai be integrated with existing blockchain applications?

   • Yes, opp/ai is designed to be flexible and can be integrated into existing applications that require both efficient and privacy-preserving AI capabilities.

Resources and Links

• opML Documentation

• Community and Support:

  • Discussion Forums: https://discord.gg/ora-io

• Related Papers and Research:

  • opML: Optimistic Machine Learning on Blockchain: https://arxiv.org/pdf/2401.17555

  • opp/ai: Optimistic Privacy-Preserving AI on Blockchain: https://arxiv.org/pdf/2402.15006

For further questions or support, please reach out to the ORA team through the community channels.

Introduction

opML (Optimistic Machine Learning) is an innovative framework that enables efficient and scalable machine learning (ML) inference directly on the blockchain. By leveraging an optimistic approach inspired by optimistic rollups (e.g., Optimism and Arbitrum), opML overcomes the computational limitations of blockchains, allowing for the execution of large ML models without sacrificing decentralization or security.

opML is currently available via ORA’s AI Oracle.

The following is a comprehensive overview of opML, including its architecture, technical components, and frequently asked questions.

Key Features of opML

• High Efficiency: Computations are performed off-chain in optimized environments and only minimal data is processed on-chain during disputes.

• Cost-Effectiveness: Reduces computational costs by avoiding the expensive proof generation required in Zero-Knowledge Machine Learning (zkML).

• Decentralization: Maintains the decentralized ethos of blockchain by enabling onchain verification without relying on centralized servers.

• Scalability: Capable of handling extensive computations that are impractical with traditional onchain methods.

• Accessibility: Opens up advanced AI models and capabilities to decentralized applications.

How opML Works

opML combines Optimistic Execution with a Fraud-Proof Mechanism.

Optimistic Execution

opML operates on the principle of optimistic execution, which assumes that the ML computations submitted are correct by default. This approach allows computations to be performed offchain in optimized environments, significantly reducing the computational burden on the blockchain.

• Assumption of Correctness: Results are accepted unless proven otherwise.

• Efficiency: By not requiring immediate verification, the system avoids unnecessary computational overhead.

Fraud-Proof Mechanism

To ensure security and correctness, opML incorporates the following fraud-proof mechanism:

1. Submission of Results: The service provider (submitter) performs the ML computation offchain and submits the result to the blockchain.

2. Verification Period: Validators (or challengers) have a predefined period (challenge period) to verify the correctness of the submitted result.

3. Dispute Resolution:

   • If a validator detects an incorrect result, they initiate an Interactive Dispute Game.

   • The dispute game efficiently pinpoints the exact computation step where the error occurred.

4. On-Chain Verification: Only the disputed computation step is verified on-chain using the Fraud Proof Virtual Machine (FPVM), minimizing resource usage.

5. Finalization: If no disputes are raised during the challenge period, or after disputes are resolved, the result is finalized on the blockchain.

Optimizations in opML

• Deterministic ML: Randomness and variability in floating-point computations results in inconsistency of ML results. To address this, opML fixes the random seed so that outputs can be verified across multiple instances of the model. ****

• Separated Execution from Proving: opML compiles the source code twice for native execution and for the fraud-proof VM instructions. This allows for computations to be performed in optimized native environments ensuring fast execution while simultaneously proving based on machine-independent code.

• Lazy Loading Design: Loads only necessary data into the FPVM during the dispute phase to overcome memory limitations of the VM.

• Attention Challenge: Validators are randomly selected to verify computations. Failing to respond when selected results in penalties, motivating active participation.

• Economic Security: Validators and submitters stake tokens to participate. Dishonest behavior results in penalties (loss of stake). By staking tokens, both submitters and validators have economic incentives to behave honestly.

Architecture

Fraud Proof Virtual Machine (FPVM)

The FPVM is a specialized virtual machine designed for opML that can:

• Execute and Verify Computation Steps: Capable of performing individual computation steps to resolve disputes.

• State Management: Uses Merkle trees to represent and manage computation states efficiently.

• Onchain Arbitration: Runs on smart contracts, enabling the on-chain verification of disputed steps.

Machine Learning Engine

The ML engine in opML is tailored for both native execution and fraud-proof scenarios:

• Dual Compilation: ML models are compiled twice:

  • Native Execution: Optimized for speed, utilizing multi-threading and GPU acceleration.

  • Fraud-Proof VM Instructions: Compiled into a machine-independent code for use in FPVM during disputes.

• Determinism and Consistency:

  • Fixed-Point Arithmetic: Ensures consistent results across different environments.

  • Software-Based Floating-Point Libraries: Guarantees cross-platform determinism.

Interactive Dispute Game

An efficient protocol for resolving disputes:

• Bisection Protocol:

  • Iteratively narrows down the computation steps to find the exact point of disagreement.

  • Reduces the amount of data and computation needed for onchain verification.

• Onchain Arbitration:

  • Only the minimal necessary computation (usually a single step) is performed onchain.

  • Ensures disputes are resolved fairly and efficiently.

Workflow

1. Request Initiation: A user (requester) submits a request for an ML inference task.

2. Computation and Submission:

   • The submitter performs the computation off-chain using the optimized ML engine.

   • Results are submitted to the blockchain.

3. Challenge Period:

   • Validators verify the correctness of the result.

   • If no disputes are raised within the predefined period, the result is accepted.

4. Dispute Resolution (if necessary):

   • A validator initiates the dispute game if an incorrect result is detected.

   • The exact computation step in dispute is identified.

   • On-chain verification is performed using the FPVM.

5. Finalization: The result is finalized after successful verification or dispute resolution.

Security Considerations

AnyTrust Assumption

• Definition: The system is secure as long as at least one validator is honest.

• Implications:

  • Even if multiple validators are malicious, a single honest validator can ensure correctness.

  • Encourages decentralization and wide participation to strengthen security.

Verifier Dilemma and Solution

• Verifier Dilemma:

  • Validators may choose not to verify computations to save resources, potentially allowing incorrect results.

• Our Solution:

  • Implementing the Attention Challenge mechanism detailed above encourages validators to participate.

  • Economic incentives ensure that the cost of cheating outweighs potential gains.

Frequently Asked Questions

1. What makes opML different from zkML?

   • opML focuses on efficiency and scalability by using an optimistic approach without zero-knowledge proofs. zkML provides strong privacy guarantees but is resource-intensive and less practical for large models. The use of opp/ai enables the use of opML with privacy guarantees.

2. How does opML ensure security without zero-knowledge proofs?

   • opML relies on a fraud-proof mechanism and the AnyTrust assumption. Validators can challenge incorrect results, and economic incentives discourage dishonest behavior.

3. What happens if no validators challenge an incorrect result?

   • The economic incentives and attention challenge mechanism are designed to ensure validators participate. However, if all validators are dishonest or fail to challenge, the system could accept incorrect results, highlighting the importance of validator participation.

4. Can opML handle private data or proprietary models?

   • opML assumes that data and models are not sensitive. For applications requiring privacy, consider using opp/ai, which integrates opML with zkML components for privacy preservation.

5. What are the hardware requirements for running opML?

   • opML is designed to run on standard PCs with CPU/GPU capabilities. No specialized hardware is required.

6. How does the dispute resolution process impact performance?

   • Disputes are rare and only involve verifying minimal computation steps on-chain. The impact on performance is minimal compared to the overall efficiency gains. The dispute resolution process can also be optimized by zkfp.

7. Is opML open-source?

   • Yes, opML is open-sourced. You can access the repository athttps://github.com/ora-io/opml. New versions will shortly be released.

Resources and Links

• opML GitHub Repository: https://github.com/ora-io/opml

• opML Paper: https://arxiv.org/pdf/2401.17555

• Issue Tracker: https://github.com/ora-io/opml/issues

For any further questions or support, feel free to reach out to the ORA team through the community channels.

Intro to RMS

RMS (Resilient Model Services) is an AI service designed to provide computation for all scenarios, ensuring resilient AI computation through the power of opML.

Use RMS

ORA API

The first stage of RMS includes ORA's AI API service that integrates seamlessly with existing AI frameworks.

API service allows developers to call upon most popular AI models for AI inference like chat completions, image generation, and more, with all interactions being decentralized and verifiable.

Use Cases

AI Agents

AI Agents within RMS can operate with high autonomy with verifiability, ensuring their operations are secure and transparent onchain.

Learn more about ORA's vision in Agent in opAgent's manifesto.

DeFAI

DeFAI, or Decentralized Finance + Artificial Intelligence, aims to abstract the complexities of DeFi with AI.

RMS will support DeFAI by providing AI services like automated trading strategies, risk management, and personal financial agent, all verifiable and decentralized.

Web3 Gaming

Web3 games on RMS can leverage AI for dynamic gameplay, NPC interactions, and real-time decision-making, all while ensuring that game logic and outcomes are verifiable on the blockchain.

Roadmap

• ✅ Stage 1 - API Service

The initial release of RMS includes the API service for AI computations, allowing developers to integrate with models like those from DeepSeek, Meta-Llama, Google, MistralAI, Qwen, and others. This stage focuses on proving the concept of decentralized and verifiable AI computation.

• Expansion of Supported Models:

Continual addition of new models and functionalities, enhancing the variety of services available through RMS.

• Integration with Onchain Protocols:

Further integration with onchain protocols to explore and implement more complex use cases such as DeFAI.

• Enhanced Framework Integration:

Developing more tools and adaption to existing libaries (eg. Unity for Web3 game developers) to utilize RMS service in compatible ways.

Why opML?

At the core of RMS is opML, a framework invented by ORA that brings verifiability and decentralization to AI computation. Using opML provides benefits of verifiability, resilience, flexibility, and transparency.

Coming Soon

RMS is part of ORA’s commitment to providing decentralized, verifiable AI solutions.

More info will be available soon.

Introduction

ERC-7641 is an extension of the standard ERC-20 token, designed to integrate a seamless revenue-sharing mechanism directly into the token's functionality. It allows token holders to claim and redeem a proportional share of a communal revenue pool, enhancing the token's utility and promoting active participation in the ecosystem.

ERC-7641 is a core component of the Initial Model Offering (IMO).

Key Benefits

• Revenue Sharing for Token Holders: Provides a standardized method for distributing revenue among token holders, rewarding them for their participation and investment.

• Funding for Projects: Enables any project, including open-source initiatives, to tokenize their revenue streams, fostering sustainable development and community engagement.

• Flexibility and Fairness: Maintains the essential attributes of ERC-20 tokens while introducing mechanisms for fair revenue distribution and deflationary economics through token burning.

How ERC-7641 Works

1. Revenue Sharing Mechanism

• Claimable Revenue: Token holders can periodically claim a portion of the revenue pool based on their token holdings at specific snapshots.

• Snapshot Functionality: Snapshots capture token balances and claimable revenue at certain points in time, ensuring accurate and fair distribution.

• Proportional Distribution: Revenue between snapshots is distributed proportionally to token holders, with a specified sharing ratio determining the portion allocated for claims.

2. Burning Mechanism

• Token Burning: Holders can burn their tokens to redeem a proportional share of the revenue pool, contributing to a deflationary model.

• Redeemable Value: The value redeemed upon burning is calculated based on the holder's share of the total token supply and the total redeemable revenue in the pool.

• Deflationary Economics: Reduces the total token supply over time, encouraging long-term holding.

ERC-7641 Interfaces

The ERC-7641 standard introduces several interfaces to facilitate its functionality:

Core Interface: IERC7641

Defines the primary functions for revenue sharing and token burning:

• claimableRevenue(address account, uint256 snapshotId): Calculates the amount of ETH claimable by a token holder at a specific snapshot.

• claim(uint256 snapshotId): Allows a token holder to claim their share of ETH based on a snapshot.

• snapshot(): Creates a new snapshot of token balances and claimable revenue, returning a unique snapshot ID.

• redeemableOnBurn(uint256 amount): Calculates the amount of ETH redeemable upon burning a specified amount of tokens.

• burn(uint256 amount): Burns a specified amount of tokens and redeems the corresponding share of the revenue pool.

Optional Extension: IERC7641AltRevToken

An optional extension to support revenue sharing with alternative ERC-20 tokens beyond ETH:

• claimableERC20(address account, uint256 snapshotId, address token): Calculates the amount of a specific ERC-20 token claimable by a holder at a snapshot.

• redeemableERC20OnBurn(uint256 amount, address token): Calculates the amount of a specific ERC-20 token redeemable upon burning tokens.

Use Cases

Funding Open-Source Projects

ERC-7641 enables open-source projects to tokenize their revenue streams, providing sustainable funding and incentivizing contributions. By distributing Intrinsic RevShare tokens to contributors, projects can reflect involvement and share revenue proportionally.

Initial Model Offering (IMO)

Combining ERC-7641 with the concept of an Initial Model Offering (IMO), open-source developers can conduct fundraising for new AI model development and create ongoing communities for their projects. Token holders benefit from revenue generated by the AI models, fostering a collaborative and sustainable ecosystem.

Additional Resources

• Reference Implementation: GitHub Repository

• ERC-7641 Proposal: Ethereum Improvement Proposal

• Related Concepts:

  • ERC-7007: Verifiable AI-Generated Content Token

  • Initial Model Offering (IMO)

Intro to ORA API

ORA API is an integral part of ORA's Resilient Model Services (RMS).

The ORA API serves as the gateway for developers to interact with RMS, providing a decentralized, verifiable, and resilient platform for AI computations.

Advantages of ORA API are the best support for the most AI models in crypto; competitive pricing and predictable onchain cost; verifiable AI inference service; OpenAI compatibility...

Integrate with ORA API

Getting Started

Before using the ORA API, developers need to:

1. Obtain API Key: Register for an ORA API key through developer portal for authenticating requests.

2. Set Up Environment: Ensure you have Python installed if you're using the SDK, or a tool like cURL for direct API calls.

Use ORA API

Chat Completion

POST https://api.ora.io/v1/chat/completions

Generate text responses from ORA API AI models.

Headers

Body

Example in shell

curl -X POST "https://api.ora.io/v1/chat/completions" \
  -H "Authorization: Bearer $ORA_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "deepseek-ai/DeepSeek-V3",
    "messages": [{"role": "user", "content": "What are some fun things to do in New York?"}]
  }'

Create a new user

POST https://api.ora.io/v1/images/generations

Generate images based on text prompts.

Headers

Body

Example in shell

curl -X POST "https://api.ora.io/v1/images/generations" \
  -H "Authorization: Bearer $ORA_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "black-forest-labs/FLUX.1-dev",
    "prompt": "Cats eating popcorn",
    "steps": 10,
    "n": 4
  }'

The generated images are stored on IPFS.

SDK Integration

ORA API is designed to be compatible with the OpenAI SDK, facilitating a smooth transition for developers already familiar with it:

import openai

# Define your query
system_content = "You are a helpful assistant."
user_content = "What are some fun things to do in New York?"

# Set your ORA API key
ORA_API_KEY = "YOUR_ORA_API_KEY"

# Initialize the client
client = openai.OpenAI(
    api_key=ORA_API_KEY,
    base_url="https://api.ora.io/v1",
)

# Perform a chat completion
chat_completion = client.chat.completions.create(
    model="deepseek-ai/DeepSeek-V3",
    messages=[
        {"role": "system", "content": system_content},
        {"role": "user", "content": user_content},
    ]
)

# Print the response
response = chat_completion.choices[0].message.content
print("Response:\n", response)

Supported Models and Pricing

ORA API supports a variety of open-source models, all verifiable through opML:

// language models
"meta-llama/Llama-3.3-70B-Instruct" = 1.76,                # Per 1M Tokens
"meta-llama/Llama-3.2-3B-Instruct" = 0.12,                 # Per 1M Tokens
"meta-llama/Meta-Llama-3-70B-Instruct" = 1.76,             # Per 1M Tokens
"meta-llama/Llama-2-13b-chat-hf" = 0.44,                   # Per 1M Tokens
"meta-llama/Llama-2-7b-chat-hf" = 0.40,                    # Per 1M Tokens
"meta-llama/Llama-3.1-405B-Instruct" = 7.0,                # Per 1M Tokens
"meta-llama/Llama-3.2-1B-Instruct" = 0.12,                 # Per 1M Tokens
"meta-llama/Meta-Llama-3-8B-Instruct" = 0.36,              # Per 1M Tokens

"google/gemma-2b-it" = 0.20,                               # Per 1M Tokens
"google/gemma-2-27b-it" = 1.60,                            # Per 1M Tokens
"google/gemma-2-9b-it" = 0.60,                             # Per 1M Tokens
"mistralai/Mistral-7B-Instruct-v0.3" = 0.40,               # Per 1M Tokens
"mistralai/Mixtral-8x22B-Instruct-v0.1" = 2.4,             # Per 1M Tokens
"mistralai/Mistral-7B-Instruct-v0.2" = 0.40,               # Per 1M Tokens
"mistralai/Mixtral-8x7B-Instruct-v0.1" = 1.20,             # Per 1M Tokens
"mistralai/Mistral-7B-Instruct-v0.1" = 0.40,               # Per 1M Tokens

"Qwen/QwQ-32B-Preview" = 2.40,                             # Per 1M Tokens
"Qwen/Qwen2.5-Coder-32B-Instruct" = 1.60,                  # Per 1M Tokens
"Qwen/Qwen2.5-72B-Instruct" = 2.40,                        # Per 1M Tokens
"Qwen/Qwen2-72B-Instruct" = 1.80,                          # Per 1M Tokens

"deepseek-ai/DeepSeek-V3" = 2.50,                          # Per 1M Tokens

// image generation models
"black-forest-labs/FLUX.1-dev" = 0.050,                    # Per 1M Pixels @ 28 Steps
"black-forest-labs/FLUX.1-canny" = 0.050,                  # Per 1M Pixels @ 28 Steps
"black-forest-labs/FLUX.1-redux-dev" = 0.050,              # Per 1M Pixels @ 28 Steps
"black-forest-labs/FLUX.1-schnell" = 0.006,                # Per 1M Pixels @ 4 Steps

"stabilityai/stable-diffusion-3.5-large" = 0.13,           # Per Image
"stabilityai/stable-diffusion-3.5-large-turbo" = 0.08,     # Per Image
"stabilityai/stable-diffusion-3-medium" = 0.07,            # Per Image
"stabilityai/stable-diffusion-3.5-medium" = 0.07,          # Per Image

Best Practices

• Error Handling: Always implement error handling to manage API response errors or timeouts.

• Rate Limiting: Be aware of and respect rate limits to avoid service disruptions.

• Security: Never expose your API key in client-side code. Use server-side calls or secure environment variables.

Overview

Initial Model Offering Platform, along with research.ora.io is the initiative to empower the development and funding of innovative AI models, to achieve blockchain intelligence with open and collaborative AI research.

The IMO process of each proposal is divided into two key stages:

• Deposit Stage: Participants contribute $ORA tokens to their chosen IMO proposal.

• Claiming Stage: Contributors to successful proposals can claim their allocated IMO tokens and any remaining $ORA tokens, if applicable.

ORA ensures that every IMO launch is decentralized and transparent, requiring $ORA as the primary deposit for allocation.

1. Review the IMO Proposal: Examine the proposal's technology and tokenomics.

2. Deposit Funds: Contribute $ORA or $ETH (converted to $ORA by the protocol) to secure an allocation and support the IMO proposal.

3. Wait for the Raising Period: Allow the raising period to conclude, during which funds are collected.

4. Claim Tokens or Refunds:

   • If the raising target is met or exceeded:

     • Claim IMO tokens and any un-utilized $ORA (for oversubscription: Allocation for IMO Tokens Per User = (Total Raising Target $ORA / Total Actual Raised $ORA) × Deposited $ORA from User).

   • If the raising target is not met:

     • Claim your original $ORA deposit in full.

Deposit Stage

During Deposit Stage, participants deposit $ORA tokens into the IMO proposal of their choice.

Once deposited, $ORA is locked until the raising period finishes, at which point the proposal is classified as either successful or failed based on expected raising amount and actual raised amount.

The following Claiming Stage will allow participants to retrieve their allocations or refunds based on the proposal's outcome.

Claiming Stage

During the Claiming Stage, participants of successful proposals can claim their $ORA tokens and $IMO tokens based on different conditions.

Successful Proposals

If the total $ORA raised exactly meets the target, participants receive solely IMO tokens based on the pricing specified in the proposal.

If the proposal is oversubscribed (raised amount exceeds the target), participants receive IMO tokens proportionally to their deposited $ORA.

Allocated $ORA for IMO Token for User
=
(Total Raising Target $ORA / Total Actual Raised $ORA)
*
Deposited $ORA of User

For example: Target = 1M $ORA; Actual Raised = 2M $ORA; Participants receive IMO tokens for 50% (= 1M / 2M) of their deposit, and the remaining 50% $ORA is refunded.

Failed Proposals

No IMO tokens are issued for failed proposals. Participants are refunded the full amount of their deposited $ORA.

FAQs

Q: What happens if a proposal is oversubscribed?

If the total $ORA raised exceeds the target, participants receive IMO tokens proportionally to their deposit. Any unconverted $ORA is refunded during the Claiming Stage.

Q: Can I deposit with $ETH during the Deposit Stage?

Yes, when depositing with $ETH, your $ETH will be auto-converted into $ORA by the protocol to secure your allocation. To maximize your deposit, using tools like Mev Blocker can help on MEV protection. In refund or oversubscription conditions, you will be able to claim $ORA that is converted from $ETH.

Q: Can I withdraw my $ORA during the Deposit Stage?

No, once deposited, $ORA remains locked until the proposal's outcome is determined.

Q: If I deposit with $ETH, what do I claim in claiming stage?

In refund or oversubscription conditions, you will be able to claim $ORA that is auto converted from $ETH in deposit stage by the protocol.

Q: What happens if a proposal fails to meet its raising target?

Participants will receive a full refund of their deposited $ORA. No IMO tokens will be issued for failed proposals.

Q: Are there any transaction fees involved?

Yes, every onchain interaction (e.g., depositing, claiming) requires Ethereum network transaction fees. Participants should ensure they have sufficient ETH for these fees.

Q: Can I participate in multiple IMO proposals?

Yes, participants can deposit $ORA into multiple proposals. Each deposit is managed independently based on the outcome of the respective proposal.

Q: How are the prices of IMO tokens determined?

The price of IMO tokens is specified in the proposal before the Deposit Stage begins. This price is fixed and forms the basis for allocation.

Q: Is there a limit to how much $ORA I can deposit?

Deposit limits, if applicable, are defined in the specific IMO proposal. If not specified, it means there's no limit on how much $ORA one can deposit. Participants should review the proposal details before depositing.

Q: Can rules vary between IMOs?

Yes, rules may vary depending on the specific IMO proposal. Participants should carefully review the details of each proposal to understand its terms.

Disclaimer

This document is intended for informational purposes only. ORA reserves the right to modify or update the rules, processes, calculations, and any other process associated with the IMO platform at any time to ensure fairness and functionality. By participating in the IMO process, you acknowledge that the final interpretation and implementation of all rules rest solely with ORA.

Models

Llama3 8B Instruct

OpenLM 1B

OpenLM Score 7B, aka. (A.I.)location Oracle

OpenLM Chat 7B

Stable Diffusion v2-1

Stable Diffusion v3

Deployed Addresses

Prompt and SimplePrompt are both example smart contracts interacted with AI Oracle.

For simpler application scenarios (eg. Prompt Engineering based AI like GPTs), you can directly use Prompt or SimplePrompt.

SimplePrompt saves gas by only emitting the event without storing historical data.

Ethereum Mainnet

Ethereum Sepolia

Deprecated contracts: AIOracle, Prompt.

Optimism Mainnet

Optimism Sepolia

Arbitrum Mainnet

Arbitrum Sepolia Testnet

Manta Mainnet

Manta Sepolia Testnet

Linea Mainnet