1 of 10

Onchain AI Oracle

An AI Oracle powered by Optimistic Machine Learning (opML)

ORA's Onchain AI Oracle is a verifiable oracle protocol that allows developers to create smart contracts and applications that ingest verifiable machine learning (ML) inferences for uses on the blockchain.

AI Oracle is powered by Optimistic Machine Learning (opML). It enables a verifiable, transparent, and decentralized way to integrate advanced ML models like LLaMA 3 and Stable Diffusion into smart contracts.

It offers:

Scalability: Can run any ML model onchain without prohibitive overhead.
Efficiency: Reduces computational time and cost when compared to zkML.
Practicality: Can easily be integrated into applications by using existing blockchain infrastructure.
Verifiability: Leverages fraud proofs to provide applications and users with assured computational integrity.

Components

AI Oracle comprises a set of smart contracts and off-chain components:

opML Contract: Handles fraud proofs and challenges to ensure on-chain verifiability of ML computations.
AIOracle Contract: Connects the opML node with on-chain callers to process ML requests and integrates various ML models.
User Contract: Customizable contracts that initiate AI inference requests and receive results from directly from AI Oracle.
ORA Nodes: Machines that run the Tora Client that interact with the ORA network. Nodes currently perform two functions: submit or validate inference results.

Build with AI Oracle

Build your dApp using ORA's AI Oracle

This tutorial will help you understand the structure of the AI Oracle, guide you through the process of building a simple Prompt contract that interacts with the ORA network.

If you prefer a video version of the tutorial, check it .

Final version of the code can be found .

Learning Objectives

Setup the development environment
Understand the project setup and template repository structure
Learn how to interact with the AI Oracle and build an AI powered smart contract

Prerequisites

To follow this tutorial you need to have and installed.

Setup

Clone template repository and install submodules

git clone -b OAO_interaction_tutorial git@github.com:ora-io/Interaction_With_OAO_Template.git --recursive

Move into the cloned repository

cd Interaction_With_OAO_Template

Copy .env.example, rename it to .env. We will need these env variables later for the deployment and testing. You can leave them empty for now.

cp .env.example .env

Install foundry dependencies

forge install

Creating Prompt contract

1. Import dependencies

At the beginning we need to import several dependencies which our smart contract will use.

import "OAO/contracts/interfaces/IAIOracle.sol";
import "OAO/contracts/AIOracleCallbackReceiver.sol";

IAIOracle - interface that defines a requestCallback method that needs to be implemented in the Prompt contract AIOracleCallbackReceiver - an abstract contract that contains an instance of AIOracle and implements a callback method that needs to be overridden in the Prompt contract

2. Write the code

We'll start by implementing the constructor, which accepts the address of the deployed AIOracle contract.

constructor(IAIOracle _aiOracle) AIOracleCallbackReceiver(_aiOracle){}

Now let’s define a method that will interact with the AI Oracle. This method requires 2 parameters, id of the model and input prompt data. It also needs to be payable, because a user needs to pass the fee for the callback execution.

function calculateAIResult(uint256 modelId, string calldata prompt) payable external {
    bytes memory input = bytes(prompt);
    bytes memory callbackData = bytes("");
    address callbackAddress = address(this);
    uint256 requestId = aiOracle.requestCallback{value: msg.value}(
        modelId, input, callbackAddress, callbackGasLimit[modelId], callbackData
    );
}

In the code above we do the following:

Convert input to bytes
Call the requestCallback function with the following parameters:
- modelId: ID of the AI model in use.
- input: User-provided prompt.
- callbackAddress: The address of the contract that will receive AI Oracle's callback.
- callbackGasLimit[modelId]: Maximum amount of that that can be spent on the callback, yet to be defined.
- callbackData: Callback data that is used in the callback.

Next step is to define the mapping that keeps track of the callback gas limit for each model and set the initial values inside the constructor. We’ll also define a modifier so that only the contract owner can change these values.

address owner;

modifier onlyOwner() {
    require(msg.sender == owner, "Only owner");
    _;
}

mapping(uint256 => uint64) public callbackGasLimit;

function setCallbackGasLimit(uint256 modelId, uint64 gasLimit) external onlyOwner {
    callbackGasLimit[modelId] = gasLimit;
}

constructor(IAIOracle _aiOracle) AIOracleCallbackReceiver(_aiOracle) {
    owner = msg.sender;
    callbackGasLimit[50] = 500_000; // Stable-Diffusion
    callbackGasLimit[11] = 5_000_000; // Llama
}

We want to store all the requests that occurred. For that we can create a data structure for the request data and a mapping between requestId and the request data.

event promptRequest(
    uint256 requestId,
    address sender, 
    uint256 modelId,
    string prompt
);

struct AIOracleRequest {
    address sender;
    uint256 modelId;
    bytes input;
    bytes output;
}

mapping(uint256 => AIOracleRequest) public requests;

function calculateAIResult(uint256 modelId, string calldata prompt) payable external {
    bytes memory input = bytes(prompt);
    bytes memory callbackData = bytes("");
    address callbackAddress = address(this);
    uint256 requestId = aiOracle.requestCallback{value: msg.value}(
        modelId, input, callbackAddress, callbackGasLimit[modelId], callbackData
    );
    AIOracleRequest storage request = requests[requestId];
    request.input = input;
    request.sender = msg.sender;
    request.modelId = modelId;
    emit promptRequest(requestId, msg.sender, modelId, prompt);
}

In the code snippet above we added prompt, sender and the modelId to the request and also emitted an event.

Now that we implemented a method for interaction with the AI Oracle, let's define a callback that will be invoked by the AI Oracle after the computation of the result.

event promptsUpdated(
    uint256 requestId,
    uint256 modelId,
    string input,
    string output,
    bytes callbackData
);

mapping(uint256 => mapping(string => string)) public prompts;

function getAIResult(uint256 modelId, string calldata prompt) external view returns (string memory) {
    return prompts[modelId][prompt];
}

function aiOracleCallback(uint256 requestId, bytes calldata output, bytes calldata callbackData) external override onlyAIOracleCallback() {
    AIOracleRequest storage request = requests[requestId];
    require(request.sender != address(0), "request does not exist");
    request.output = output;
    prompts[request.modelId][string(request.input)] = string(output);
    emit promptsUpdated(requestId, request.modelId, string(request.input), string(output), callbackData);
}

We've overridden the callback function from the AIOracleCallbackReceiver.sol. It is important to use the modifier, so that only AI Oracle can callback into our contract.

Function flow:

First we check if the request with provided id exists. If it does we add the output value to the request.
Then we define prompts mapping that stores all the prompts and outputs for each model that we use.
At the end we emit an event that the prompt has been updated.

Notice that this function takes callbackData as the last parameter. This parameter can be used to execute arbitrary logic during the callback. It is passed during requestCallback call. In our simple example, we left it empty. Finally let's add the method that will estimate the fee for the callback call.

function estimateFee(uint256 modelId) public view returns (uint256) {
    return aiOracle.estimateFee(modelId, callbackGasLimit[modelId]);
}

With this, we've completed with the source code for our contract. The final version should look like this:

// SPDX-License-Identifier: MIT
pragma solidity ^0.8.13;

import "OAO/contracts/interfaces/IAIOracle.sol";
import "OAO/contracts/AIOracleCallbackReceiver.sol";

contract Prompt is AIOracleCallbackReceiver {
    
    event promptsUpdated(
        uint256 requestId,
        uint256 modelId,
        string input,
        string output,
        bytes callbackData
    );

    event promptRequest(
        uint256 requestId,
        address sender, 
        uint256 modelId,
        string prompt
    );

    struct AIOracleRequest {
        address sender;
        uint256 modelId;
        bytes input;
        bytes output;
    }

    address owner;

    modifier onlyOwner() {
        require(msg.sender == owner, "Only owner");
        _;
    }

    mapping(uint256 => AIOracleRequest) public requests;

    mapping(uint256 => uint64) public callbackGasLimit;

    constructor(IAIOracle _aiOracle) AIOracleCallbackReceiver(_aiOracle) {
        owner = msg.sender;
        callbackGasLimit[50] = 500_000; // Stable-Diffusion
        callbackGasLimit[11] = 5_000_000; // Llama
    }

    function setCallbackGasLimit(uint256 modelId, uint64 gasLimit) external onlyOwner {
        callbackGasLimit[modelId] = gasLimit;
    }

    mapping(uint256 => mapping(string => string)) public prompts;

    function getAIResult(uint256 modelId, string calldata prompt) external view returns (string memory) {
        return prompts[modelId][prompt];
    }

    function aiOracleCallback(uint256 requestId, bytes calldata output, bytes calldata callbackData) external override onlyAIOracleCallback() {
        AIOracleRequest storage request = requests[requestId];
        require(request.sender != address(0), "request does not exist");
        request.output = output;
        prompts[request.modelId][string(request.input)] = string(output);
        emit promptsUpdated(requestId, request.modelId, string(request.input), string(output), callbackData);
    }

    function estimateFee(uint256 modelId) public view returns (uint256) {
        return aiOracle.estimateFee(modelId, callbackGasLimit[modelId]);
    }

    function calculateAIResult(uint256 modelId, string calldata prompt) payable external {
        bytes memory input = bytes(prompt);
        bytes memory callbackData = bytes("");
        address callbackAddress = address(this);
        uint256 requestId = aiOracle.requestCallback{value: msg.value}(
            modelId, input, callbackAddress, callbackGasLimit[modelId], callbackData
        );
        AIOracleRequest storage request = requests[requestId];
        request.input = input;
        request.sender = msg.sender;
        request.modelId = modelId;
        emit promptRequest(requestId, msg.sender, modelId, prompt);
    }
}

Deployment and interaction with the contract

Foundry version

Add your PRIVATE_KEY, RPC_URL and ETHERSCAN_KEY to .env file. Then source variables in the terminal.
```
source .env
```

Create a deployment script

Then open script/Prompt.s.sol and add the following code:

// SPDX-License-Identifier: MIT
pragma solidity ^0.8.13;

import {Script} from "forge-std/Script.sol";
import {Prompt} from "../src/Prompt.sol";
import {IAIOracle} from "OAO/contracts/interfaces/IAIOracle.sol";

contract PromptScript is Script {
    address OAO_PROXY;

    function setUp() public {
        OAO_PROXY = [OAO_PROXY_address_here];
    }

    function run() public {
        uint privateKey = vm.envUint("PRIVATE_KEY");
        vm.startBroadcast(privateKey);
        new Prompt(IAIOracle(OAO_PROXY));
        vm.stopBroadcast();
    }
}

Run the deployment script

forge script script/Prompt.s.sol --rpc-url $RPC_URL --broadcast --verify --etherscan-api-key $ETHERSCAN_KEY

Let's use Stable Diffusion model (id = 50) as an example.
1. First call estimateFee method to calculate fee for the callback.
2. Request the AI inference from the AI Oracle by calling calculateAIResult method. Pass the model id and the prompt for the image generation. Remember to provide an estimated fee as a value for the transaction.

Remix version

Install the browser wallet if you haven't already (eg. Metamask)
Copy the contract along with necessary dependencies to Remix.
Choose the solidity compiler version and compile the contract to the bytecode
Deploy the compiled bytecode Once we compiled our contract, we can deploy it.
1. First go to the wallet and choose the blockchain network for the deployment.
3. Deploy the contract by signing the transaction in the wallet.

Conclusion

In this tutorial we covered a step by step writing of a solidity smart contract that interacts with ORA's AI Oracle. Then we compiled and deployed our contract to the live network and interacted with it. In the next tutorial, we will extend the functionality of the Prompt contract to support AI generated NFT collections.

Resources

Callback Gas Limit Estimation

When interacting with the AI Oracle, it is crucial to allocate sufficient gas for the execution of the callback function. The gas consumed during the callback depends entirely on the implementation of the aiOracleCallback method. Its logic directly impacts gas usage - the more complex the logic, the higher the gas consumption. Carefully design your aiOracleCallback to balance functionality and gas efficiency.

To estimate the gas required for the aiOracleCallback in a Foundry project:

Run the following command to generate a gas report:
```
forge snapshot --gas-report
```
Locate your Prompt contract in the report and check the gas usage for the aiOracleCallback method.
Use the calculated gas amount to set the gas limit during the initialisation of the Prompt contract (in the constructor).

Alternatively you can use other tools like Hardhat or Tenderly.

Test Script

We provided for gas limit estimation in the template repository.
To execute estimation run: forge test test/EstimateGasLimit.t.sol -vvvv . Next to the callback function, you can observe the gas amount needed for execution. eg. [1657555] Prompt::aiOracleCallback -> in this case amount we can set gas limit to 1700000

// SPDX-License-Identifier: UNLICENSED
pragma solidity ^0.8.13;

import {Test, console2, Vm} from "forge-std/Test.sol";
import {Prompt} from "../src/Prompt.sol";
import {IAIOracle} from "OAO/contracts/interfaces/IAIOracle.sol";
import {OraSepoliaAddresses} from "./OraSepoliaAddresses.t.sol";
import "forge-std/console.sol";

contract EstimateGasLimitTest is Test, OraSepoliaAddresses {
    Prompt prompt;
    string rpc;
    uint256 forkId;
    uint256 modelId;
    string result;

    function setUp() public {
        rpc = vm.envString("RPC_URL");
        forkId = vm.createSelectFork(rpc);
        prompt = new Prompt(IAIOracle(OAO_PROXY));
        modelId = 50;
        result = "Qmd3xWJVRao8AfgRTuXLCWBYj6VdVV8HFuKygk9FLTW5bi";
    }

    function test_estimateGasLimit() public {
        uint256 requestId = prompt.calculateAIResult{value: prompt.estimateFee(modelId)}(modelId, "test generation");
        uint256 before = gasleft();
        vm.prank(OAO_PROXY);
        prompt.aiOracleCallback(requestId, bytes(result), "");
        uint256 afterCall = gasleft();
        console.log(before - afterCall);
    }
}

The above script estimates gas necessary for Stable-Diffusion callback. Note that result will always be constant size (ipfs CID).

Llama3

To estimate gas limit for Llama3, you need to change modelId and result.

modelId = 11;
result = "Beneath the sky, where whispers blend, A story stirs that has no end. The trees bow low, the rivers hum, A timeless tune, where life has sprung. The sun ascends with golden rays, To bless the fields and mark the days. The flowers bloom, their colors speak, Of beauty vast, profound, unique. The stars alight, a cosmic sea, Of dreams untold and destiny. Each twinkle holds a secret dear, A tale of love, of hope, of fear. Through valleys deep and mountains wide, The journey calls, the hearts true guide. With every step, a lesson found, In fleeting moments, wisdoms ground. Yet in the shadows, doubts may creep, A restless thought, a troubled sleep. But courage whispers, soft and clear, Youre not alone; Im always near. So carry forth, embrace the storm, Through fire and frost, the heart is warm. For lifes a dance, both wild and free, A fleeting song of harmony. As time unfurls its endless thread, The soul will soar where dreams are led. A symphony of lifes embrace, endless journey through boundless space. Beneath the moon, the rivers gleam, A quiet whisper shapes a dream. Through winds of change and skies of blue, I love you.";

💡 The maximum length of Llama3 response is 200 words string, hence we will use this string size to estimate gas limit.

Advanced Usages of AI Oracle

In this tutorial, we’ll explore advanced techniques for interacting with the AI Oracle. Specifically, we’ll dive into topics like , , and Data Availability (DA) Options, giving you a deeper understanding of these features and how to leverage them effectively.

1. Nested Inference

A user can perform nested inference by initiating a second inference based on the result of the first inference within a smart contract. This action can be completed atomically and is not restricted to a two-step function.

Nested Inference Use Cases

Some of the use cases for a nested inference call include:

generating a prompt with LLM for AIGC (AI Generated Content) NFT
extracting data from a data set, then generate visual data with different models
adding transcript to a video, then translate it to different languages with different models

For demo purposes we built a that uses ORA's AI Oracle.

Implementing Nested Inference

The idea of Nested Inference contract is to execute multiple inference requests in 1 transaction. We'll modify contract to support nested inference request. In our example, it will call Llama3 model first, then use inference result as the prompt to another request to StableDiffusion model.

The main goal of this tutorial is to understand what changes we need to make to Prompt contract in order to implement logic for various use cases.

Implementation Steps

modify CalculateAIResult method to support multiple requests
modify aiOracleCallback with the logic to handle second inference request

💡 When estimating gas cost for the callback, we should take both models into the consideration.

CalculateAIResult

As we now have additional function parameter for second model id. Not that we encode and forward model2Id as a callback data in aiOracle.requestCallback call.

 function calculateAIResult(uint256 model1Id, uint256 model2Id, string calldata model1Prompt) payable external returns (uint256) {
    bytes memory input = bytes(model1Prompt);
    uint256 model1Fee = estimateFee(model1Id);
    uint256 requestId = aiOracle.requestCallback{value: model1Fee}(
        model1Id, input, address(this), callbackGasLimit[model1Id], abi.encode(model2Id)
    );
    AIOracleRequest storage request = requests[requestId];
    request.input = input;
    request.sender = msg.sender;
    request.modelId = model1Id;
    emit promptRequest(requestId, msg.sender, model1Id, model1Prompt);
    return requestId;
}

aiOracleCallback

The main change here is the within "if" block. If the callback data (model2Id) is returned, we want to execute second inference request to the AI Oracle.

Output from the first inference call, will be passed to second one. This allows for interesting use cases, where you can combine text-to-text (eg. Llama3) and text-to-image (eg. Stable-Diffusion) models.

If nested inference call is not successful the whole function will revert.

💡 When interacting with the contract from the client side, we need to pass cumulative fee (for both models), then for each inference call we need to pass part of that cumulative fee. This is why we are calling estimateFee for model2Id.

function aiOracleCallback(uint256 requestId, bytes calldata output, bytes calldata callbackData) external payable override onlyAIOracleCallback() {
    AIOracleRequest storage request = requests[requestId];
    require(request.sender != address(0), "request does not exist");
    request.output = output;
    prompts[request.modelId][string(request.input)] = string(output);

    //if callbackData is not empty decode it and call another inference
    if(callbackData.length != 0){
        (uint256 model2Id) = abi.decode(callbackData, (uint256));
        uint256 model2Fee = estimateFee(model2Id);

        (bool success, bytes memory data) = address(aiOracle).call{value: model2Fee}(abi.encodeWithSignature("requestCallback(uint256,bytes,address,uint64,bytes)", model2Id, output, address(this), callbackGasLimit[model2Id], ""));
        require(success, "failed to call nested inference");

        (uint256 rid) = abi.decode(data, (uint256));
        AIOracleRequest storage recursiveRequest = requests[rid];
        recursiveRequest.input = output;
        recursiveRequest.sender = msg.sender;
        recursiveRequest.modelId = model2Id;
        emit promptRequest(rid, msg.sender, model2Id, "");
    }

    emit promptsUpdated(requestId, request.modelId, string(request.input), string(output), callbackData);
}

Interaction with Contract

This is an example of contract interaction from Foundry testing environment. Note that we're estimating fee for both models and passing cumulative amount during the function call (we're passing slightly more to ensure that the call will execute if the gas price changes).

PromptNestedInference prompt = new PromptNestedInference(IAIOracle(OAO_PROXY));

uint256 stableDiffusionFee = prompt.estimateFee(STABLE_DIFFUSION_ID);
uint256 llamaFee = prompt.estimateFee(LLAMA_ID);

uint256 requestId = prompt.calculateAIResult{value: ((stableDiffusionFee + llamaFee)*11/10)}(STABLE_DIFFUSION_ID, LLAMA_ID, SD_PROMPT);

Conclusion

2. Batch Inference

The Batch Inference feature enables sending multiple inference requests within a single transaction, reducing costs by saving on network fees and improving the user experience. This bulk processing allows for more efficient handling of requests and results, making it easier to manage the state of multiple queries simultaneously.

Some of the use cases might be:

AIGC NFT marketplace - creating a whole AIGC NFT collection with just one transaction, instead of creating those with many transactions
Apps that need to handle requests simultaneously - Good example would be a recommendation system or chatbot with high TPS

Pricing

Callback transaction fee is needed for AI Oracle to submit callback transaction. It's calculated by multiplying current gas price with callback gas limit for invoked model (gasPrice * callbackGasLimit).

Request transaction fee is regular blockchain fee needed to request inference by invoking aiOracle.requestCallback method.

Total fee is calculated as sum of Model fee, Callback transaction fee and Request transaction fee.

Implementing Batch Inference

In order to initiate batch inference request, we will interact with requestBatchInference method. This method takes additional batchSize parameter, which specifies the amount of requests to the AI Oracle.

function calculateAIResult(uint256 modelId, string calldata prompt, uint256 batchSize) payable external {
    bytes memory input = bytes(prompt);
    bytes memory callbackData = bytes("");
    address callbackAddress = address(this);
    uint256 requestId = aiOracle.requestBatchInference{value: msg.value}(
        batchSize, modelId, input, callbackAddress, callbackGasLimit[modelId], callbackData, IAIOracle.DA(0), IAIOracle.DA(0)
    );
}

//Prompt.sol
function estimateFeeBatch(uint256 modelId) public view returns (uint256) {
    return aiOracle.estimateFee(modelId, callbackGasLimit[modelId]);
}

//AIOracle.sol
function estimateFeeBatch(uint256 modelId, uint256 gasLimit, uint256 batchSize) public view ifModelExists(modelId) returns (uint256) {
    ModelData storage model = models[modelId];
    return batchSize * model.fee + gasPrice * gasLimit;
}

Prompt Format

Input for batch inference should be structured as a string representing an array of prompt and seed values.

Prompt - string value that is mandatory in order to prompt AI Oracle.

Seed – an optional numeric value that, when used, allows you to generate slightly varied responses for the same prompt.

[
    // Batch inference requests with required "prompt" and optional "seed" or "seed_range".
    {
        "prompt": "prompt1",  // Mandatory prompt
        "seed": 1             // Optional seed for response variation
    },
    {
        "prompt": "prompt2",
        "seed": 2
    },
    {
        "prompt": "prompt3"    // No seed provided, default behavior
    },
    {
        "prompt": "prompt4",
        "seed_range": [1, 2]   // Seed range for controlled variation
    }
]

Output Format

Result of Batch inference call is a dot separated list of inference results.

result1.result2.result3...

Example

This is the prompt for interacting with Stable Diffusion model:

[{"prompt":"generate image of an elephant","seed":1},{"prompt":"generate image of an elephant","seed":2}]

Result is dot separated list of ipfs CIDs:

.Qma3gR6z9dtSQ3fsoSUwQtHnNMYZEHhuJx3DBqx677HPaj.QmemPcnzdyigx1XnAvskVKmEhnsivBuE3Qkp3jJqfdWWxC

Interaction with Batch Inference

import { Web3 } from "web3";

const web3 = new Web3(process.env.RPC_URL);

const wallet = web3.eth.accounts.wallet.add(process.env.PRIVATE_KEY); // Make sure you have funds

const contract = new web3.eth.Contract(batchInference_abi, batchInference_address);

const prompt = `[{"prompt":"what's the best fruit", "seed":1},{"prompt":"what's the best fruit", "seed":2},{"prompt":"kiwi"},{"prompt":"dog", "seed_range":[1,2]}]`

const fee = Number(await contract.methods.estimateFeeBatch(11, 5).call());

const totalFee = (fee*11/10)

const tx = await contract.methods.requestBatchInference(11, prompt, 5).send({from: wallet[0].address, value: totalFee});
console.log("Tx: ", tx)

setTimeout(async () => {
    const result = await contract.methods.prompts(11, prompt).call();
    console.log("Result: ", result)
}, 30000);

To test it, you need to:

create new javascript file
copy the script and add env variables
create and deploy prompt contract that supports batch inference
add values to batchInference_abi and batchInference_address

Prompt examples

// Example 1: Batch with identical prompts but different seeds for varied responses
const prompt1 = `[{"prompt":"hello","seed":1},{"prompt":"hello","seed":2}]`

// Example 2: Batch with different prompts, no seeds specified, uses default behavior
const prompt2 = `[{"prompt":"hello"},{"prompt":"hello world"}]`

// Example 3: Batch with same question but varied seeds, and different prompts with a seed range
const prompt3 = `[{"prompt":"what's the best fruit", "seed":1},{"prompt":"what's the best fruit", "seed":2},{"prompt":"kiwi"},{"prompt":"dog","seed_range":[1,2]}]`

// Example 4: Batch with identical prompt but different seeds, plus a seed range for a specific prompt
const prompt4 = `[{"prompt":"apple", "seed":1}, {"prompt":"apple", "seed":2}, {"prompt":"banana"}, {"prompt":"cat", "seed_range": [1,100] }]`

Conclusion

Optimistic Machine Learning (OPML)

Introduction

The Optimistic Machine Learning (OPML) framework connects off-chain machine learning computations with smart contracts. Results are provably correct and can be disputed on-chain to ensure trust and accountability, enabling efficient AI-driven decision-making in decentralized applications. OPML is a core component of the AI Oracle, playing a critical role in validating inference results.

Workflow

Initiate Request: The user contract sends an inference request to AI Oracle by calling the requestCallback function.
opML Request: AI Oracle creates an opML request based on the user contract request.
Event Emission: AI Oracle emits a requestCallback event collected by the opML node.
ML Inference: The opML node performs the AI model computation.
Result Submission: The opML node uploads the result on-chain.
Callback Execution: The result is dispatched to the user's smart contract via a callback function.

Challenge Process

Challenge Window: Begins after the result is submitted on-chain (step 5 above).
Verification: opML validators or any network participant can check the results and challenge the output if it is incorrect.
Result Update: If a challenge is successful, the incorrect result is updated on-chain.
Finality: After the challenge period, the result is finalized onchain and made immutable.

Onboard AI Models

Bring Your Own Model

In this tutorial we explain how to integrate your own AI model with ORA's AI Oracle. We will start by looking at repository and trying to understand what's happening there. At the end we will showcase how works, by running a simple dispute game script inside a docker container.

Learning Objectives

Understand how to transform an AI model inference code and integrate it with ORA's AI Oracle
Execute a simple dispute game and understand the process of AI inference verification.

Prerequisites

installed

Setup

Clone repository

git clone git@github.com:OPML-Labs/mlgo.git

Navigate to cloned repository

cd mlgo

To install the required dependencies for your project, run the following command:

pip install -r requirements.txt

If there are some missing dependencies, make sure to install them in your Python environment.

Train the model using Pytorch

First we need to train a DNN model using Pytorch. The training part is shown in examples/mnist/trainning/mnist.ipynb.

After the training model is saved at examples/mnist/models/mnist/mnist-small.state_dict.

Convert model into ggml format

Position to mnist folder

cd examples/mnist

Convert python model into ggml format

python3 convert-h5-to-ggml.py models/mnist/mnist-small.state_dict

In order to convert AI model written in Python to ggml format we are executing python script and providing a file that stores the model as a parameter to the script. The output is the binary file in ggml format. Note that the model is saved in big-endian, making it easy to process in the big-endian MIPS-32 VM.

Converting inference code to MIPS VM executable

Next step is to write inference code in go language. Then we will transform go binary into MIPS VM executable file.

Go supports compilation to MIPS. However, the generated executable is in ELF format. We'd like to get a pure sequence of MIPS instructions instead. To build a ML program in MIPS VM execute the following steps:

Navigate to the mnist_mips directory and build go inference code

cd ../mnist_mips && ./build.sh

Running the dispute game

Now that we compiled our AI model and inference code into MIPS VM executable code.

First we need to specify the operating system that runs inside our container. In this case we're using ubuntu:22.04.

# How to run instructions:
# 1. Generate ssh command: ssh-keygen -t rsa -b 4096 -C "your_email@example.com"
#    - Save the key in local repo where Dockerfile is placed as id_rsa
#    - Add the public key to the GitHub account
# 2. Build docker image: docker build -t ubuntu-opml-dev .
# 3. Run the hardhat: docker run -it --rm --name ubuntu-opml-dev-container ubuntu-opml-dev bash -c "npx hardhat node"
# 4. Run the challange script on the same container: docker exec -it ubuntu-opml-dev-container bash -c "./demo/challenge_simple.sh"


# Use an official Ubuntu as a parent image
FROM ubuntu:22.04

# Set environment variables to non-interactive to avoid prompts during package installations
ENV DEBIAN_FRONTEND=noninteractive

# Update the package list and install dependencies
RUN apt-get update && apt-get install -y \
    build-essential \
    cmake \
    git \
    golang \
    wget \
    curl \
    python3 \
    python3-pip \
    python3-venv \
    unzip \
    file \
    openssh-client \
    && apt-get clean \
    && rm -rf /var/lib/apt/lists/*

# Install Node.js and npm
RUN curl -fsSL https://deb.nodesource.com/setup_18.x | bash - && \
    apt-get install -y nodejs

Then we configure ssh keys, so that docker container can clone all the required repositories.

# Copy SSH keys into the container
COPY id_rsa /root/.ssh/id_rsa
RUN chmod 600 /root/.ssh/id_rsa
# Configure SSH to skip host key verification
RUN echo "Host *\n\tStrictHostKeyChecking no\n" >> /root/.ssh/config

Position to the root directory inside docker container and clone opml repository along with its submodules.

# Set the working directory
WORKDIR /root

# Clone the OPML repository
RUN git clone git@github.com:ora-io/opml.git --recursive
WORKDIR /root/opml

Lastly, we tell docker to build executables and run the challenge script.

# Build the OPML project
RUN make build

# Change permission for the challenge script
RUN chmod +x demo/challenge_simple.sh

# Default command
CMD ["bash"]

Create docker container and run the script

In order to successfully clone opml repository, you need to generate a new ssh key and add it to your Github account. Once it's generated, save the key in the local repository where Dockerfile is placed as id_rsa. Then add the public key to your GitHub account.
```
ssh-keygen -t rsa -b 4096 -C "your_email@example.com"
```
Build the docker image
```
docker build -t ubuntu-opml-dev .
```

Run the local Ethereum node

docker run -it --rm --name ubuntu-opml-dev-container ubuntu-opml-dev bash -c "npx hardhat node"

In another terminal run the challenge script

docker exec -it ubuntu-opml-dev-container bash -c "./demo/challenge_simple.sh"

After executing the steps above you should be able to see interactive challenge process in the console.

Script first deploys necessary contracts to the local node. Proposer opML node executes AI inference and then the challenger nodes can dispute it, if they think that the result is not valid. Challenger and proposer are interacting in order to find the differing step between their computations. Once the dispute step is found it's sent to the smart contract for the arbitration. If the challenge is successful the proposer node gets slashed.

In this tutorial we achieved the following:

converted our AI model from Python to ggml format
compiled AI inference code written in go to MIPS VM executable format
run the dispute game inside a docker container and understood the opML verification process

Non-Developer Guides

Easy to use guides for non-developers

Using AI Oracle

As a user, to interact with AI Oracle, you can:

Use the AI.ORA.IO frontend- view our video tutorial
Directly via the Prompt contract on Etherscan

Via Etherscan

In Prompt contract's Read Contract section, call estimateFee with specified modelId.

In Prompt contract's Write Contract section, call calculateAIResult with fee (converted from wei to ether), prompt, and modelId.

In the AI Oracle contract, look for the new transaction that fulfills the request you sent.

In Prompt contract's Read Contract section, call getAIResult with previous modelId and prompt to view the AI inference result.

Retrieve Historical AI Inference

If you want to retrieve your historical AI inference (eg. AIGC image), you can find them on blockchain explorer:

Find your transaction for sending AI request, and enter the "Logs" tab. Example tx: https://etherscan.io/tx/0xfbfdb2efcee23197c5ea8487368a905385c84afdc465cab43bc1ad01da773404#eventlog
Access your requestId in "Logs" tab Example's case: "1928".
In OAO's smart contract, look for the Invoke Callback transaction with the same requestId. Normally, this transaction will be around the same time as the one in step 1. To filter transactions by date, click "Advanced Filter" and then the button near "Age".
Find the transaction for AI inference result, and enter the "Logs" tab. Example tx: https://etherscan.io/tx/0xfbfdb2efcee23197c5ea8487368a905385c84afdc465cab43bc1ad01da773404#eventlog
Access output data. Example's case: "QmecBGR7dD7pRtY48FEKoeLVsmBTLwvdicWRkX9xz2NVvC" which is the IPFS hash that can be accessed with IPFS gateway)

References

Supported AI Models and Deployment Addresses of ORA's AI Oracle.

Models

Llama3 8B Instruct

Model ID

Deployed Network

Ethereum & Ethereum Sepolia, Optimism & Optimism Sepolia, Arbitrum & Arbitrum Sepolia, Manta & Manta Sepolia, Linea, Base, Mantle, Polygon PoS

Fee

Mainnet: 0.0003 ETH / 3 MATIC / 3 MNT Testnet: 0.01 ETH

Usage

OpenLM 1B

Model ID

Deployed Network

Ethereum & Ethereum Sepolia

Fee

Mainnet: 0.0003 ETH Testnet: 0.01 ETH

Usage

OpenLM Score 7B, aka. (A.I.)location Oracle

Model ID

Deployed Network

Arbitrum & Arbitrum Sepolia, Ethereum Sepolia

Fee

Mainnet: 0.0003 ETH Testnet: 0.01 ETH

Usage

Note

Input exceeds 7000 characters won't receive callback.

OpenLM Chat 7B

OpenLM Score 7B, aka. (A.I.)location Oracle

Model ID

Deployed Network

Arbitrum & Arbitrum Sepolia, Ethereum Sepolia

Fee

Mainnet: 0.0003 ETH Testnet: 0.01 ETH

Usage

Stable Diffusion v2-1

Model ID

Deployed Network

Ethereum & Ethereum Sepolia, Optimism & Optimism Sepolia, Arbitrum & Arbitrum Sepolia, Manta & Manta Sepolia, Linea, Base, Mantle, Polygon PoS

Fee

Mainnet: 0.0003 ETH / 3 MATIC / 3 MNT Testnet: 0.01 ETH

Usage

Stable Diffusion v3

Stable Diffusion v3 Medium

Model ID

503

Deployed Network

Ethereum & Ethereum Sepolia, Optimism & Optimism Sepolia, Arbitrum & Arbitrum Sepolia, Manta & Manta Sepolia, Linea, Base, Mantle, Polygon PoS

Fee

Mainnet: 0.0003 ETH / 3 MATIC / 3 MNT Testnet: 0.01 ETH

Usage

ORA RMS API Models

ORA RMS API Models are supported on Ethereum mainnet and Base networks.

| Model Name                                  | Price per Call in $ORA |
|---------------------------------------------|------------------------|
| meta-llama/Llama-3.3-70B-Instruct           | 0.18                   |
| Qwen/QwQ-32B-Preview                        | 0.24                   |
| Qwen/Qwen2.5-Coder-32B-Instruct             | 0.16                   |
| meta-llama/Llama-3.2-3B-Instruct            | 0.01                   |
| mistralai/Mixtral-8x22B-Instruct-v0.1       | 0.24                   |
| meta-llama/Meta-Llama-3-70B-Instruct        | 0.18                   |
| Qwen/Qwen2-72B-Instruct                     | 0.18                   |
| google/gemma-2-27b-it                       | 0.16                   |
| google/gemma-2-9b-it                        | 0.06                   |
| mistralai/Mistral-7B-Instruct-v0.3          | 0.04                   |
| google/gemma-2b-it                          | 0.02                   |
| mistralai/Mistral-7B-Instruct-v0.2          | 0.04                   |
| mistralai/Mixtral-8x7B-Instruct-v0.1        | 0.12                   |
| mistralai/Mistral-7B-Instruct-v0.1          | 0.04                   |
| meta-llama/Llama-2-13b-chat-hf              | 0.04                   |
| meta-llama/Llama-2-7b-chat-hf               | 0.04                   |
| meta-llama/Llama-3.1-405B-Instruct          | 0.7                    |
| Qwen/Qwen2.5-72B-Instruct                   | 0.24                   |
| meta-llama/Llama-3.2-1B-Instruct            | 0.01                   |
| meta-llama/Meta-Llama-3-8B-Instruct         | 0.04                   |
| black-forest-labs/FLUX.1-dev                | 0.09                   |
| black-forest-labs/FLUX.1-canny              | 0.09                   |
| black-forest-labs/FLUX.1-redux-dev          | 0.09                   |
| black-forest-labs/FLUX.1-schnell            | 0.01                   |
| deepseek-ai/DeepSeek-V3                     | 0.25                   |
| stabilityai/stable-diffusion-3.5-medium     | 0.13                   |
| stabilityai/stable-diffusion-3-medium       | 0.13                   |
| stabilityai/stable-diffusion-3.5-large      | 0.24                   |
| stabilityai/stable-diffusion-3.5-large-turbo| 0.15                   |

To determine the model ID of a specific model, please refer to the code below:

// solidity
function calcModelIdByName(string calldata modelName) public pure returns (uint256) {
    return uint256(uint160(uint256(keccak256(bytes(modelName)))));
}

// JavaScript
// example modelIdString: 'openai/gpt-4o'
function modelIdStringToBigInt(modelIdString) {
  const hashedValue = ethers.keccak256(ethers.toUtf8Bytes(modelIdString));
  const addressValue = `0x${hashedValue.slice(26)}`;
  return BigInt(addressValue);
}

Deployed Addresses

Prompt and SimplePrompt are both example smart contracts interacted with AI Oracle.

For simpler application scenarios (eg. Prompt Engineering based AI like GPTs), you can directly use Prompt or SimplePrompt.

SimplePrompt saves gas by only emitting the event without storing historical data.

Ethereum Mainnet

Ethereum Sepolia

Deprecated contracts: AIOracle, Prompt.

Optimism Mainnet

Optimism Sepolia

Arbitrum Mainnet

Arbitrum Sepolia Testnet

Manta Mainnet

Manta Sepolia Testnet

Linea Mainnet

Base Mainnet

Base Sepolia

Polygon PoS Mainnet

Mantle Mainnet

Morph Mainnet

AI Settlement Oracle

Introduction

The AI Settlement Oracle is the first AI-powered truth machine, leveraging verifiable AI to resolve and settle factual questions onchain. It offers a trustless, autonomous system for information resolution, eliminating human error and manipulation.

Advantages

Autonomous Operation: Powered by ORA’s Optimistic Machine Learning (OPML) and AI Oracle.
Accuracy & Fairness: Immune to economic manipulation and herd behaviour typical of traditional systems.
Trustless Settlement: Ensures unbiased outcomes for any factual query.

Try it out!

Live Demo
Github Repo
Research Post

Architecture

Workflow

The AI Settlement Oracle contains two parts of interaction:

Parse Context: The initial interaction processes a question by analyzing web sources and summarizing them into a coherent "truth context." This provides a reliable foundation for further reasoning.
Onchain Settlement: In the second interaction, the truth context and helper prompts are integrated into an input for the AI Oracle. The Oracle uses this input to generate a verified outcome, enabling decentralized and transparent truth settlement onchain.

Example: Fortune Teller

To showcase how AI Oracle can enhance consumer-facing products, we introduce .

About Fortune Teller

Fortune Teller leverages the NestedInference smart contract, as discussed in the . This application aims to onboard new users to Web3 and demonstrate the capabilities of .

Application Flow

User Interaction: The application prompts users to answer specific questions posed by the Magic Wizard.
Fortune Telling: The Wizard casts its spells by requesting inferences from AI Oracle.
Response Generation: The Llama3 model generates a textual response, which is then used as a prompt for image creation via StableDiffusion.
NFT Minting: Users can mint AI-generated NFTs if they are satisfied with their fortune image.

The image on the right represents user's fortune result generated by AI Oracle:

Benefits of using AI Oracle in Various Applications

Objective AI-Generated Insights: AI provides neutral and unbiased outputs, ensuring a fair experience across diverse use cases.
Immutable Onchain Data: Information generated is securely stored on the blockchain, making it tamper-proof and easily verifiable.
Transparent Data Generation: Utilizing opML, the entire generation process is transparent, fostering trust in the system across different applications.

Build your own AI dapps!

Build with AI Oracle

Build your dApp using ORA's AI Oracle

This tutorial will help you understand the structure of the AI Oracle, guide you through the process of building a simple Prompt contract that interacts with the ORA network.

If you prefer a video version of the tutorial, check it .

Final version of the code can be found .

Learning Objectives

Setup the development environment
Understand the project setup and template repository structure
Learn how to interact with the AI Oracle and build an AI powered smart contract

Prerequisites

To follow this tutorial you need to have and installed.

Setup

Clone template repository and install submodules

git clone -b OAO_interaction_tutorial git@github.com:ora-io/Interaction_With_OAO_Template.git --recursive

Move into the cloned repository

cd Interaction_With_OAO_Template

Copy .env.example, rename it to .env. We will need these env variables later for the deployment and testing. You can leave them empty for now.

cp .env.example .env

Install foundry dependencies

forge install

Creating Prompt contract

1. Import dependencies

At the beginning we need to import several dependencies which our smart contract will use.

import "OAO/contracts/interfaces/IAIOracle.sol";
import "OAO/contracts/AIOracleCallbackReceiver.sol";

2. Write the code

We'll start by implementing the constructor, which accepts the address of the deployed AIOracle contract.

constructor(IAIOracle _aiOracle) AIOracleCallbackReceiver(_aiOracle){}

function calculateAIResult(uint256 modelId, string calldata prompt) payable external {
    bytes memory input = bytes(prompt);
    bytes memory callbackData = bytes("");
    address callbackAddress = address(this);
    uint256 requestId = aiOracle.requestCallback{value: msg.value}(
        modelId, input, callbackAddress, callbackGasLimit[modelId], callbackData
    );
}

In the code above we do the following:

Convert input to bytes
Call the requestCallback function with the following parameters:
- modelId: ID of the AI model in use.
- input: User-provided prompt.
- callbackAddress: The address of the contract that will receive AI Oracle's callback.
- callbackGasLimit[modelId]: Maximum amount of that that can be spent on the callback, yet to be defined.
- callbackData: Callback data that is used in the callback.

address owner;

modifier onlyOwner() {
    require(msg.sender == owner, "Only owner");
    _;
}

mapping(uint256 => uint64) public callbackGasLimit;

function setCallbackGasLimit(uint256 modelId, uint64 gasLimit) external onlyOwner {
    callbackGasLimit[modelId] = gasLimit;
}

constructor(IAIOracle _aiOracle) AIOracleCallbackReceiver(_aiOracle) {
    owner = msg.sender;
    callbackGasLimit[50] = 500_000; // Stable-Diffusion
    callbackGasLimit[11] = 5_000_000; // Llama
}

💡 Gas Limit Estimation - It's important to define callbackGasLimit for each model to ensure that AIOracle will have enough funds to return the response! Read more on page.

We want to store all the requests that occurred. For that we can create a data structure for the request data and a mapping between requestId and the request data.

event promptRequest(
    uint256 requestId,
    address sender, 
    uint256 modelId,
    string prompt
);

struct AIOracleRequest {
    address sender;
    uint256 modelId;
    bytes input;
    bytes output;
}

mapping(uint256 => AIOracleRequest) public requests;

function calculateAIResult(uint256 modelId, string calldata prompt) payable external {
    bytes memory input = bytes(prompt);
    bytes memory callbackData = bytes("");
    address callbackAddress = address(this);
    uint256 requestId = aiOracle.requestCallback{value: msg.value}(
        modelId, input, callbackAddress, callbackGasLimit[modelId], callbackData
    );
    AIOracleRequest storage request = requests[requestId];
    request.input = input;
    request.sender = msg.sender;
    request.modelId = modelId;
    emit promptRequest(requestId, msg.sender, modelId, prompt);
}

In the code snippet above we added prompt, sender and the modelId to the request and also emitted an event.

Now that we implemented a method for interaction with the AI Oracle, let's define a callback that will be invoked by the AI Oracle after the computation of the result.

event promptsUpdated(
    uint256 requestId,
    uint256 modelId,
    string input,
    string output,
    bytes callbackData
);

mapping(uint256 => mapping(string => string)) public prompts;

function getAIResult(uint256 modelId, string calldata prompt) external view returns (string memory) {
    return prompts[modelId][prompt];
}

function aiOracleCallback(uint256 requestId, bytes calldata output, bytes calldata callbackData) external override onlyAIOracleCallback() {
    AIOracleRequest storage request = requests[requestId];
    require(request.sender != address(0), "request does not exist");
    request.output = output;
    prompts[request.modelId][string(request.input)] = string(output);
    emit promptsUpdated(requestId, request.modelId, string(request.input), string(output), callbackData);
}

We've overridden the callback function from the AIOracleCallbackReceiver.sol. It is important to use the modifier, so that only AI Oracle can callback into our contract.

Function flow:

First we check if the request with provided id exists. If it does we add the output value to the request.
Then we define prompts mapping that stores all the prompts and outputs for each model that we use.
At the end we emit an event that the prompt has been updated.

function estimateFee(uint256 modelId) public view returns (uint256) {
    return aiOracle.estimateFee(modelId, callbackGasLimit[modelId]);
}

With this, we've completed with the source code for our contract. The final version should look like this:

// SPDX-License-Identifier: MIT
pragma solidity ^0.8.13;

import "OAO/contracts/interfaces/IAIOracle.sol";
import "OAO/contracts/AIOracleCallbackReceiver.sol";

contract Prompt is AIOracleCallbackReceiver {
    
    event promptsUpdated(
        uint256 requestId,
        uint256 modelId,
        string input,
        string output,
        bytes callbackData
    );

    event promptRequest(
        uint256 requestId,
        address sender, 
        uint256 modelId,
        string prompt
    );

    struct AIOracleRequest {
        address sender;
        uint256 modelId;
        bytes input;
        bytes output;
    }

    address owner;

    modifier onlyOwner() {
        require(msg.sender == owner, "Only owner");
        _;
    }

    mapping(uint256 => AIOracleRequest) public requests;

    mapping(uint256 => uint64) public callbackGasLimit;

    constructor(IAIOracle _aiOracle) AIOracleCallbackReceiver(_aiOracle) {
        owner = msg.sender;
        callbackGasLimit[50] = 500_000; // Stable-Diffusion
        callbackGasLimit[11] = 5_000_000; // Llama
    }

    function setCallbackGasLimit(uint256 modelId, uint64 gasLimit) external onlyOwner {
        callbackGasLimit[modelId] = gasLimit;
    }

    mapping(uint256 => mapping(string => string)) public prompts;

    function getAIResult(uint256 modelId, string calldata prompt) external view returns (string memory) {
        return prompts[modelId][prompt];
    }

    function aiOracleCallback(uint256 requestId, bytes calldata output, bytes calldata callbackData) external override onlyAIOracleCallback() {
        AIOracleRequest storage request = requests[requestId];
        require(request.sender != address(0), "request does not exist");
        request.output = output;
        prompts[request.modelId][string(request.input)] = string(output);
        emit promptsUpdated(requestId, request.modelId, string(request.input), string(output), callbackData);
    }

    function estimateFee(uint256 modelId) public view returns (uint256) {
        return aiOracle.estimateFee(modelId, callbackGasLimit[modelId]);
    }

    function calculateAIResult(uint256 modelId, string calldata prompt) payable external {
        bytes memory input = bytes(prompt);
        bytes memory callbackData = bytes("");
        address callbackAddress = address(this);
        uint256 requestId = aiOracle.requestCallback{value: msg.value}(
            modelId, input, callbackAddress, callbackGasLimit[modelId], callbackData
        );
        AIOracleRequest storage request = requests[requestId];
        request.input = input;
        request.sender = msg.sender;
        request.modelId = modelId;
        emit promptRequest(requestId, msg.sender, modelId, prompt);
    }
}

Deployment and interaction with the contract

Foundry version

Add your PRIVATE_KEY, RPC_URL and ETHERSCAN_KEY to .env file. Then source variables in the terminal.
```
source .env
```

Create a deployment script

Go to page and find the OAO_PROXY address for the network you want to deploy to.

Then open script/Prompt.s.sol and add the following code:

// SPDX-License-Identifier: MIT
pragma solidity ^0.8.13;

import {Script} from "forge-std/Script.sol";
import {Prompt} from "../src/Prompt.sol";
import {IAIOracle} from "OAO/contracts/interfaces/IAIOracle.sol";

contract PromptScript is Script {
    address OAO_PROXY;

    function setUp() public {
        OAO_PROXY = [OAO_PROXY_address_here];
    }

    function run() public {
        uint privateKey = vm.envUint("PRIVATE_KEY");
        vm.startBroadcast(privateKey);
        new Prompt(IAIOracle(OAO_PROXY));
        vm.stopBroadcast();
    }
}

Run the deployment script

forge script script/Prompt.s.sol --rpc-url $RPC_URL --broadcast --verify --etherscan-api-key $ETHERSCAN_KEY

Once the contract is deployed and verified, you can interact with it. Go to blockchain explorer for your chosen network (eg. ), and paste the address of the deployed Prompt contract.
Let's use Stable Diffusion model (id = 50) as an example.
1. First call estimateFee method to calculate fee for the callback.
2. Request the AI inference from the AI Oracle by calling calculateAIResult method. Pass the model id and the prompt for the image generation. Remember to provide an estimated fee as a value for the transaction.
3. After the transaction is executed, and the AI Oracle calculates the result, you can check it by calling prompts method. Simply input model id and the prompt you used for image generation. In the case of Stable Diffusion, the output will be a CID (content identifier on ). To check the image go to .

Remix version

Install the browser wallet if you haven't already (eg. Metamask)
Open your solidity development environment. We'll use .
Copy the contract along with necessary dependencies to Remix.
Choose the solidity compiler version and compile the contract to the bytecode
Deploy the compiled bytecode Once we compiled our contract, we can deploy it.
1. First go to the wallet and choose the blockchain network for the deployment.
2. To deploy the Prompt contract we need to provide the address of already deployed AIOracle contract. You can find this address on the . We are looking for OAO_PROXY address.
3. Deploy the contract by signing the transaction in the wallet.
Once the contract is deployed, you can interact with it. Remix supports API for interaction, but you can also use blockchain explorers like . Let's use Stable Diffusion model (id = 50).
1. First call estimateFee method to calculate fee for the callback.
2. Then request AI inference from the AI Oracle by calling `calculateAIResult` method. Pass the model id and the prompt for the image generation. Remember to provide estimated fee as a value for the transaction.
3. After the transaction is executed, and the AI Oracle calculates result, you can check it by calling prompts method. Simply input model id and the prompt you used for image generation. In the case of Stable Diffusion, the output will be a CID (content identifier on ).

Conclusion

Resources

Source code of AI Oracle:

For supported models and deployment addresses, see page.

For example integrations and ideas to build, see .

Check out AI oracle.

Advanced Usages of AI Oracle

1. Nested Inference

Nested Inference Use Cases

Some of the use cases for a nested inference call include:

generating a prompt with LLM for AIGC (AI Generated Content) NFT
extracting data from a data set, then generate visual data with different models
adding transcript to a video, then translate it to different languages with different models

For demo purposes we built a that uses ORA's AI Oracle.

Implementing Nested Inference

The main goal of this tutorial is to understand what changes we need to make to Prompt contract in order to implement logic for various use cases.

Implementation Steps

modify CalculateAIResult method to support multiple requests
modify aiOracleCallback with the logic to handle second inference request

💡 When estimating gas cost for the callback, we should take both models into the consideration.

CalculateAIResult

As we now have additional function parameter for second model id. Not that we encode and forward model2Id as a callback data in aiOracle.requestCallback call.

 function calculateAIResult(uint256 model1Id, uint256 model2Id, string calldata model1Prompt) payable external returns (uint256) {
    bytes memory input = bytes(model1Prompt);
    uint256 model1Fee = estimateFee(model1Id);
    uint256 requestId = aiOracle.requestCallback{value: model1Fee}(
        model1Id, input, address(this), callbackGasLimit[model1Id], abi.encode(model2Id)
    );
    AIOracleRequest storage request = requests[requestId];
    request.input = input;
    request.sender = msg.sender;
    request.modelId = model1Id;
    emit promptRequest(requestId, msg.sender, model1Id, model1Prompt);
    return requestId;
}

aiOracleCallback

The main change here is the within "if" block. If the callback data (model2Id) is returned, we want to execute second inference request to the AI Oracle.

If nested inference call is not successful the whole function will revert.

function aiOracleCallback(uint256 requestId, bytes calldata output, bytes calldata callbackData) external payable override onlyAIOracleCallback() {
    AIOracleRequest storage request = requests[requestId];
    require(request.sender != address(0), "request does not exist");
    request.output = output;
    prompts[request.modelId][string(request.input)] = string(output);

    //if callbackData is not empty decode it and call another inference
    if(callbackData.length != 0){
        (uint256 model2Id) = abi.decode(callbackData, (uint256));
        uint256 model2Fee = estimateFee(model2Id);

        (bool success, bytes memory data) = address(aiOracle).call{value: model2Fee}(abi.encodeWithSignature("requestCallback(uint256,bytes,address,uint64,bytes)", model2Id, output, address(this), callbackGasLimit[model2Id], ""));
        require(success, "failed to call nested inference");

        (uint256 rid) = abi.decode(data, (uint256));
        AIOracleRequest storage recursiveRequest = requests[rid];
        recursiveRequest.input = output;
        recursiveRequest.sender = msg.sender;
        recursiveRequest.modelId = model2Id;
        emit promptRequest(rid, msg.sender, model2Id, "");
    }

    emit promptsUpdated(requestId, request.modelId, string(request.input), string(output), callbackData);
}

Interaction with Contract

PromptNestedInference prompt = new PromptNestedInference(IAIOracle(OAO_PROXY));

uint256 stableDiffusionFee = prompt.estimateFee(STABLE_DIFFUSION_ID);
uint256 llamaFee = prompt.estimateFee(LLAMA_ID);

uint256 requestId = prompt.calculateAIResult{value: ((stableDiffusionFee + llamaFee)*11/10)}(STABLE_DIFFUSION_ID, LLAMA_ID, SD_PROMPT);

Conclusion

You can also check the full implementation of .

2. Batch Inference

Some of the use cases might be:

AIGC NFT marketplace - creating a whole AIGC NFT collection with just one transaction, instead of creating those with many transactions
Apps that need to handle requests simultaneously - Good example would be a recommendation system or chatbot with high TPS

Pricing

Model fee required for batch inference request is calculated by multiplying single model fee with batch size (batchSize * model.fee). This fee covers the operational costs of running AI models, with a portion contributing to protocol revenue. For details on the required fees for each model, visit the page.

Request transaction fee is regular blockchain fee needed to request inference by invoking aiOracle.requestCallback method.

Total fee is calculated as sum of Model fee, Callback transaction fee and Request transaction fee.

Implementing Batch Inference

function calculateAIResult(uint256 modelId, string calldata prompt, uint256 batchSize) payable external {
    bytes memory input = bytes(prompt);
    bytes memory callbackData = bytes("");
    address callbackAddress = address(this);
    uint256 requestId = aiOracle.requestBatchInference{value: msg.value}(
        batchSize, modelId, input, callbackAddress, callbackGasLimit[modelId], callbackData, IAIOracle.DA(0), IAIOracle.DA(0)
    );
}

Note that we'll need to pass more gas to cover AI Oracle callback execution, depending on the batchSize (check out ). For this purpose we can implement estimateFeeBatch function in our contract. This method will interact with estimateFeeBatch method from AIOracle.sol.

//Prompt.sol
function estimateFeeBatch(uint256 modelId) public view returns (uint256) {
    return aiOracle.estimateFee(modelId, callbackGasLimit[modelId]);
}

//AIOracle.sol
function estimateFeeBatch(uint256 modelId, uint256 gasLimit, uint256 batchSize) public view ifModelExists(modelId) returns (uint256) {
    ModelData storage model = models[modelId];
    return batchSize * model.fee + gasPrice * gasLimit;
}

Prompt Format

Input for batch inference should be structured as a string representing an array of prompt and seed values.

Prompt - string value that is mandatory in order to prompt AI Oracle.

Seed – an optional numeric value that, when used, allows you to generate slightly varied responses for the same prompt.

[
    // Batch inference requests with required "prompt" and optional "seed" or "seed_range".
    {
        "prompt": "prompt1",  // Mandatory prompt
        "seed": 1             // Optional seed for response variation
    },
    {
        "prompt": "prompt2",
        "seed": 2
    },
    {
        "prompt": "prompt3"    // No seed provided, default behavior
    },
    {
        "prompt": "prompt4",
        "seed_range": [1, 2]   // Seed range for controlled variation
    }
]

Output Format

Result of Batch inference call is a dot separated list of inference results.

result1.result2.result3...

Example

This is the prompt for interacting with Stable Diffusion model:

[{"prompt":"generate image of an elephant","seed":1},{"prompt":"generate image of an elephant","seed":2}]

Result is dot separated list of ipfs CIDs:

.Qma3gR6z9dtSQ3fsoSUwQtHnNMYZEHhuJx3DBqx677HPaj.QmemPcnzdyigx1XnAvskVKmEhnsivBuE3Qkp3jJqfdWWxC

Interaction with Batch Inference

When performing batch inference with the AI Oracle, ensure the prompt is following the . Below is a simple script for interacting with batch inference:

import { Web3 } from "web3";

const web3 = new Web3(process.env.RPC_URL);

const wallet = web3.eth.accounts.wallet.add(process.env.PRIVATE_KEY); // Make sure you have funds

const contract = new web3.eth.Contract(batchInference_abi, batchInference_address);

const prompt = `[{"prompt":"what's the best fruit", "seed":1},{"prompt":"what's the best fruit", "seed":2},{"prompt":"kiwi"},{"prompt":"dog", "seed_range":[1,2]}]`

const fee = Number(await contract.methods.estimateFeeBatch(11, 5).call());

const totalFee = (fee*11/10)

const tx = await contract.methods.requestBatchInference(11, prompt, 5).send({from: wallet[0].address, value: totalFee});
console.log("Tx: ", tx)

setTimeout(async () => {
    const result = await contract.methods.prompts(11, prompt).call();
    console.log("Result: ", result)
}, 30000);

To test it, you need to:

create new javascript file
copy the script and add env variables
create and deploy prompt contract that supports batch inference
add values to batchInference_abi and batchInference_address

Prompt examples

// Example 1: Batch with identical prompts but different seeds for varied responses
const prompt1 = `[{"prompt":"hello","seed":1},{"prompt":"hello","seed":2}]`

// Example 2: Batch with different prompts, no seeds specified, uses default behavior
const prompt2 = `[{"prompt":"hello"},{"prompt":"hello world"}]`

// Example 3: Batch with same question but varied seeds, and different prompts with a seed range
const prompt3 = `[{"prompt":"what's the best fruit", "seed":1},{"prompt":"what's the best fruit", "seed":2},{"prompt":"kiwi"},{"prompt":"dog","seed_range":[1,2]}]`

// Example 4: Batch with identical prompt but different seeds, plus a seed range for a specific prompt
const prompt4 = `[{"prompt":"apple", "seed":1}, {"prompt":"apple", "seed":2}, {"prompt":"banana"}, {"prompt":"cat", "seed_range": [1,100] }]`

Conclusion

That's it! With few simple changes to the contract we utilised batch inference feature. This allowed us to get multiple responses with only one transaction.