In this tutorial we explain how to integrate your own AI model with ORA's AI Oracle. We will start by looking at mlgo repository and trying to understand what's happening there. At the end we will showcase how opML works, by running a simple dispute game script inside a docker container.

Learning Objectives

• Understand how to transform an AI model inference code and integrate it with ORA's AI Oracle

• Execute a simple dispute game and understand the process of AI inference verification.

Prerequisites

• git installed

Setup

1. Clone mlgo repository

git clone git@github.com:OPML-Labs/mlgo.git

1. Navigate to cloned repository

cd mlgo

1. To install the required dependencies for your project, run the following command:

pip install -r requirements.txt

If there are some missing dependencies, make sure to install them in your Python environment.

Train the model using Pytorch

First we need to train a DNN model using Pytorch. The training part is shown in examples/mnist/trainning/mnist.ipynb.

After the training model is saved at examples/mnist/models/mnist/mnist-small.state_dict.

Convert model into ggml format

Ggml is a file format that consists of version number, followed by three components that define a large language model: the model's hyperparameters, its vocabulary, and its weights. Ggml allows for more efficient inference runs on CPU. We will now convert the Python model to ggml format by executing following steps:

1. Position to mnist folder

cd examples/mnist

1. Convert python model into ggml format

python3 convert-h5-to-ggml.py models/mnist/mnist-small.state_dict

In order to convert AI model written in Python to ggml format we are executing python script and providing a file that stores the model as a parameter to the script. The output is the binary file in ggml format. Note that the model is saved in big-endian, making it easy to process in the big-endian MIPS-32 VM.

Converting inference code to MIPS VM executable

Next step is to write inference code in go language. Then we will transform go binary into MIPS VM executable file.

Go supports compilation to MIPS. However, the generated executable is in ELF format. We'd like to get a pure sequence of MIPS instructions instead. To build a ML program in MIPS VM execute the following steps:

1. Navigate to the mnist_mips directory and build go inference code

cd ../mnist_mips && ./build.sh

Build script will compile go code and then run compile.py script that will transform compiled go code to the MIPS VM executable file.

Running the dispute game

Now that we compiled our AI model and inference code into MIPS VM executable code.

We can test the dispute game process. We will use a bash script from opml repository to showcase the whole verification flow.

For this part of the tutorial we will use Docker, so make sure to have it installed.

Let's first check the content of the Dockerfile that we are using:

1. First we need to specify the operating system that runs inside our container. In this case we're using ubuntu:22.04.

# How to run instructions:
# 1. Generate ssh command: ssh-keygen -t rsa -b 4096 -C "your_email@example.com"
#    - Save the key in local repo where Dockerfile is placed as id_rsa
#    - Add the public key to the GitHub account
# 2. Build docker image: docker build -t ubuntu-opml-dev .
# 3. Run the hardhat: docker run -it --rm --name ubuntu-opml-dev-container ubuntu-opml-dev bash -c "npx hardhat node"
# 4. Run the challange script on the same container: docker exec -it ubuntu-opml-dev-container bash -c "./demo/challenge_simple.sh"


# Use an official Ubuntu as a parent image
FROM ubuntu:22.04

1. Then we need to install all the necessary dependencies in order to run dispute game script.

# Set environment variables to non-interactive to avoid prompts during package installations
ENV DEBIAN_FRONTEND=noninteractive

# Update the package list and install dependencies
RUN apt-get update && apt-get install -y \
    build-essential \
    cmake \
    git \
    golang \
    wget \
    curl \
    python3 \
    python3-pip \
    python3-venv \
    unzip \
    file \
    openssh-client \
    && apt-get clean \
    && rm -rf /var/lib/apt/lists/*

# Install Node.js and npm
RUN curl -fsSL https://deb.nodesource.com/setup_18.x | bash - && \
    apt-get install -y nodejs

1. Then we configure ssh keys, so that docker container can clone all the required repositories.

# Copy SSH keys into the container
COPY id_rsa /root/.ssh/id_rsa
RUN chmod 600 /root/.ssh/id_rsa
# Configure SSH to skip host key verification
RUN echo "Host *\n\tStrictHostKeyChecking no\n" >> /root/.ssh/config

1. Position to the root directory inside docker container and clone opml repository along with its submodules.

# Set the working directory
WORKDIR /root

# Clone the OPML repository
RUN git clone git@github.com:ora-io/opml.git --recursive
WORKDIR /root/opml

1. Lastly, we tell docker to build executables and run the challenge script.

# Build the OPML project
RUN make build

# Change permission for the challenge script
RUN chmod +x demo/challenge_simple.sh

# Default command
CMD ["bash"]

Create docker container and run the script

1. In order to successfully clone opml repository, you need to generate a new ssh key and add it to your Github account. Once it's generated, save the key in the local repository where Dockerfile is placed as id_rsa. Then add the public key to your GitHub account.

   Copy

   ssh-keygen -t rsa -b 4096 -C "your_email@example.com"

2. Build the docker image

   Copy

   docker build -t ubuntu-opml-dev .

3. Run the local Ethereum node

   Copy

   docker run -it --rm --name ubuntu-opml-dev-container ubuntu-opml-dev bash -c "npx hardhat node"

4. In another terminal run the challenge script

   Copy

   docker exec -it ubuntu-opml-dev-container bash -c "./demo/challenge_simple.sh"

After executing the steps above you should be able to see interactive challenge process in the console.

Script first deploys necessary contracts to the local node. Proposer opML node executes AI inference and then the challenger nodes can dispute it, if they think that the result is not valid. Challenger and proposer are interacting in order to find the differing step between their computations. Once the dispute step is found it's sent to the smart contract for the arbitration. If the challenge is successful the proposer node gets slashed.

In this tutorial we achieved the following:

• converted our AI model from Python to ggml format

• compiled AI inference code written in go to MIPS VM executable format

• run the dispute game inside a docker container and understood the opML verification process

When interacting with the AI Oracle, it is crucial to allocate sufficient gas for the execution of the callback function. The gas consumed during the callback depends entirely on the implementation of the aiOracleCallback method. Its logic directly impacts gas usage - the more complex the logic, the higher the gas consumption. Carefully design your aiOracleCallback to balance functionality and gas efficiency.

To estimate the gas required for the aiOracleCallback in a Foundry project:

1. Run the following command to generate a gas report:

   Copy

   forge snapshot --gas-report

2. Locate your Prompt contract in the report and check the gas usage for the aiOracleCallback method.

3. Use the calculated gas amount to set the gas limit during the initialisation of the Prompt contract (in the constructor).

Alternatively you can use other tools like Hardhat or Tenderly.

This tutorial will help you understand the structure of the AI Oracle, guide you through the process of building a simple Prompt contract that interacts with the ORA network.

If you prefer a video version of the tutorial, check it here.

Final version of the code can be found here.

Learning Objectives

• Setup the development environment

• Understand the project setup and template repository structure

• Learn how to interact with the AI Oracle and build an AI powered smart contract

Prerequisites

To follow this tutorial you need to have Foundry and git installed.

Setup

1. Clone template repository and install submodules

git clone -b OAO_interaction_tutorial git@github.com:ora-io/Interaction_With_OAO_Template.git --recursive

1. Move into the cloned repository

cd Interaction_With_OAO_Template

1. Copy .env.example, rename it to .env. We will need these env variables later for the deployment and testing. You can leave them empty for now.

cp .env.example .env

1. Install foundry dependencies

forge install

Creating Prompt contract

1. Import dependencies

At the beginning we need to import several dependencies which our smart contract will use.

import "OAO/contracts/interfaces/IAIOracle.sol";
import "OAO/contracts/AIOracleCallbackReceiver.sol";

IAIOracle - interface that defines a requestCallback method that needs to be implemented in the Prompt contract AIOracleCallbackReceiver - an abstract contract that contains an instance of AIOracle and implements a callback method that needs to be overridden in the Prompt contract

2. Write the code

We'll start by implementing the constructor, which accepts the address of the deployed AIOracle contract.

constructor(IAIOracle _aiOracle) AIOracleCallbackReceiver(_aiOracle){}

Now let’s define a method that will interact with the AI Oracle. This method requires 2 parameters, id of the model and input prompt data. It also needs to be payable, because a user needs to pass the fee for the callback execution.

function calculateAIResult(uint256 modelId, string calldata prompt) payable external {
    bytes memory input = bytes(prompt);
    bytes memory callbackData = bytes("");
    address callbackAddress = address(this);
    uint256 requestId = aiOracle.requestCallback{value: msg.value}(
        modelId, input, callbackAddress, callbackGasLimit[modelId], callbackData
    );
}

In the code above we do the following:

1. Convert input to bytes

2. Call the requestCallback function with the following parameters:

   • modelId: ID of the AI model in use.

   • input: User-provided prompt.

   • callbackAddress: The address of the contract that will receive AI Oracle's callback.

   • callbackGasLimit[modelId]: Maximum amount of that that can be spent on the callback, yet to be defined.

   • callbackData: Callback data that is used in the callback.

Next step is to define the mapping that keeps track of the callback gas limit for each model and set the initial values inside the constructor. We’ll also define a modifier so that only the contract owner can change these values.

address owner;

modifier onlyOwner() {
    require(msg.sender == owner, "Only owner");
    _;
}

mapping(uint256 => uint64) public callbackGasLimit;

function setCallbackGasLimit(uint256 modelId, uint64 gasLimit) external onlyOwner {
    callbackGasLimit[modelId] = gasLimit;
}

constructor(IAIOracle _aiOracle) AIOracleCallbackReceiver(_aiOracle) {
    owner = msg.sender;
    callbackGasLimit[50] = 500_000; // Stable-Diffusion
    callbackGasLimit[11] = 5_000_000; // Llama
}

💡 Gas Limit Estimation - It's important to define callbackGasLimit for each model to ensure that AIOracle will have enough funds to return the response! Read more on Callback gas limit Estimation page.

We want to store all the requests that occurred. For that we can create a data structure for the request data and a mapping between requestId and the request data.

event promptRequest(
    uint256 requestId,
    address sender, 
    uint256 modelId,
    string prompt
);

struct AIOracleRequest {
    address sender;
    uint256 modelId;
    bytes input;
    bytes output;
}

mapping(uint256 => AIOracleRequest) public requests;

function calculateAIResult(uint256 modelId, string calldata prompt) payable external {
    bytes memory input = bytes(prompt);
    bytes memory callbackData = bytes("");
    address callbackAddress = address(this);
    uint256 requestId = aiOracle.requestCallback{value: msg.value}(
        modelId, input, callbackAddress, callbackGasLimit[modelId], callbackData
    );
    AIOracleRequest storage request = requests[requestId];
    request.input = input;
    request.sender = msg.sender;
    request.modelId = modelId;
    emit promptRequest(requestId, msg.sender, modelId, prompt);
}

In the code snippet above we added prompt, sender and the modelId to the request and also emitted an event.

Now that we implemented a method for interaction with the AI Oracle, let's define a callback that will be invoked by the AI Oracle after the computation of the result.

event promptsUpdated(
    uint256 requestId,
    uint256 modelId,
    string input,
    string output,
    bytes callbackData
);

mapping(uint256 => mapping(string => string)) public prompts;

function getAIResult(uint256 modelId, string calldata prompt) external view returns (string memory) {
    return prompts[modelId][prompt];
}

function aiOracleCallback(uint256 requestId, bytes calldata output, bytes calldata callbackData) external override onlyAIOracleCallback() {
    AIOracleRequest storage request = requests[requestId];
    require(request.sender != address(0), "request does not exist");
    request.output = output;
    prompts[request.modelId][string(request.input)] = string(output);
    emit promptsUpdated(requestId, request.modelId, string(request.input), string(output), callbackData);
}

We've overridden the callback function from the AIOracleCallbackReceiver.sol. It is important to use the modifier, so that only AI Oracle can callback into our contract.

Function flow:

1. First we check if the request with provided id exists. If it does we add the output value to the request.

2. Then we define prompts mapping that stores all the prompts and outputs for each model that we use.

3. At the end we emit an event that the prompt has been updated.

Notice that this function takes callbackData as the last parameter. This parameter can be used to execute arbitrary logic during the callback. It is passed during requestCallback call. In our simple example, we left it empty. Finally let's add the method that will estimate the fee for the callback call.

function estimateFee(uint256 modelId) public view returns (uint256) {
    return aiOracle.estimateFee(modelId, callbackGasLimit[modelId]);
}

With this, we've completed with the source code for our contract. The final version should look like this:

// SPDX-License-Identifier: MIT
pragma solidity ^0.8.13;

import "OAO/contracts/interfaces/IAIOracle.sol";
import "OAO/contracts/AIOracleCallbackReceiver.sol";

contract Prompt is AIOracleCallbackReceiver {
    
    event promptsUpdated(
        uint256 requestId,
        uint256 modelId,
        string input,
        string output,
        bytes callbackData
    );

    event promptRequest(
        uint256 requestId,
        address sender, 
        uint256 modelId,
        string prompt
    );

    struct AIOracleRequest {
        address sender;
        uint256 modelId;
        bytes input;
        bytes output;
    }

    address owner;

    modifier onlyOwner() {
        require(msg.sender == owner, "Only owner");
        _;
    }

    mapping(uint256 => AIOracleRequest) public requests;

    mapping(uint256 => uint64) public callbackGasLimit;

    constructor(IAIOracle _aiOracle) AIOracleCallbackReceiver(_aiOracle) {
        owner = msg.sender;
        callbackGasLimit[50] = 500_000; // Stable-Diffusion
        callbackGasLimit[11] = 5_000_000; // Llama
    }

    function setCallbackGasLimit(uint256 modelId, uint64 gasLimit) external onlyOwner {
        callbackGasLimit[modelId] = gasLimit;
    }

    mapping(uint256 => mapping(string => string)) public prompts;

    function getAIResult(uint256 modelId, string calldata prompt) external view returns (string memory) {
        return prompts[modelId][prompt];
    }

    function aiOracleCallback(uint256 requestId, bytes calldata output, bytes calldata callbackData) external override onlyAIOracleCallback() {
        AIOracleRequest storage request = requests[requestId];
        require(request.sender != address(0), "request does not exist");
        request.output = output;
        prompts[request.modelId][string(request.input)] = string(output);
        emit promptsUpdated(requestId, request.modelId, string(request.input), string(output), callbackData);
    }

    function estimateFee(uint256 modelId) public view returns (uint256) {
        return aiOracle.estimateFee(modelId, callbackGasLimit[modelId]);
    }

    function calculateAIResult(uint256 modelId, string calldata prompt) payable external {
        bytes memory input = bytes(prompt);
        bytes memory callbackData = bytes("");
        address callbackAddress = address(this);
        uint256 requestId = aiOracle.requestCallback{value: msg.value}(
            modelId, input, callbackAddress, callbackGasLimit[modelId], callbackData
        );
        AIOracleRequest storage request = requests[requestId];
        request.input = input;
        request.sender = msg.sender;
        request.modelId = modelId;
        emit promptRequest(requestId, msg.sender, modelId, prompt);
    }
}

Deployment and interaction with the contract

Foundry version

1. Add your PRIVATE_KEY, RPC_URL and ETHERSCAN_KEY to .env file. Then source variables in the terminal.

   Copy

   source .env

2. Create a deployment script

   Go to Reference page and find the OAO_PROXY address for the network you want to deploy to.

   Then open script/Prompt.s.sol and add the following code:

   Copy

   // SPDX-License-Identifier: MIT
   pragma solidity ^0.8.13;
   
   import {Script} from "forge-std/Script.sol";
   import {Prompt} from "../src/Prompt.sol";
   import {IAIOracle} from "OAO/contracts/interfaces/IAIOracle.sol";
   
   contract PromptScript is Script {
       address OAO_PROXY;
   
       function setUp() public {
           OAO_PROXY = [OAO_PROXY_address_here];
       }
   
       function run() public {
           uint privateKey = vm.envUint("PRIVATE_KEY");
           vm.startBroadcast(privateKey);
           new Prompt(IAIOracle(OAO_PROXY));
           vm.stopBroadcast();
       }
   }

3. Run the deployment script

   Copy

   forge script script/Prompt.s.sol --rpc-url $RPC_URL --broadcast --verify --etherscan-api-key $ETHERSCAN_KEY

4. Once the contract is deployed and verified, you can interact with it. Go to blockchain explorer for your chosen network (eg. Etherscan), and paste the address of the deployed Prompt contract.

   Let's use Stable Diffusion model (id = 50) as an example.

   1. First call estimateFee method to calculate fee for the callback.
   2. Request the AI inference from the AI Oracle by calling calculateAIResult method. Pass the model id and the prompt for the image generation. Remember to provide an estimated fee as a value for the transaction.
   3. After the transaction is executed, and the AI Oracle calculates the result, you can check it by calling prompts method. Simply input model id and the prompt you used for image generation. In the case of Stable Diffusion, the output will be a CID (content identifier on ipfs). To check the image go to https://ipfs.io/ipfs/[Replace_your_CID].

Remix version

1. Install the browser wallet if you haven't already (eg. Metamask)

2. Open your solidity development environment. We'll use Remix IDE.

3. Copy the contract along with necessary dependencies to Remix.
4. Choose the solidity compiler version and compile the contract to the bytecode
5. Deploy the compiled bytecode Once we compiled our contract, we can deploy it.

   1. First go to the wallet and choose the blockchain network for the deployment.

   2. To deploy the Prompt contract we need to provide the address of already deployed AIOracle contract. You can find this address on the reference page. We are looking for OAO_PROXY address.

   3. Deploy the contract by signing the transaction in the wallet.
6. Once the contract is deployed, you can interact with it. Remix supports API for interaction, but you can also use blockchain explorers like Etherscan. Let's use Stable Diffusion model (id = 50).

Conclusion

In this tutorial we covered a step by step writing of a solidity smart contract that interacts with ORA's AI Oracle. Then we compiled and deployed our contract to the live network and interacted with it. In the next tutorial, we will extend the functionality of the Prompt contract to support AI generated NFT collections.

Resources

Source code of AI Oracle: https://github.com/ora-io/OAO

For supported models and deployment addresses, see Reference page.

For example integrations and ideas to build, see awesome-ora.

Check out video tutorial on interacting and building with AI oracle.

Models

Llama3 8B Instruct

OpenLM 1B

OpenLM Score 7B, aka. (A.I.)location Oracle

OpenLM Chat 7B

Stable Diffusion v2-1

Stable Diffusion v3

Deployed Addresses

Prompt and SimplePrompt are both example smart contracts interacted with AI Oracle.

For simpler application scenarios (eg. Prompt Engineering based AI like GPTs), you can directly use Prompt or SimplePrompt.

SimplePrompt saves gas by only emitting the event without storing historical data.

Ethereum Mainnet

Ethereum Sepolia

Deprecated contracts: AIOracle, Prompt.

Optimism Mainnet

Optimism Sepolia

Arbitrum Mainnet

Arbitrum Sepolia Testnet

Manta Mainnet

Manta Sepolia Testnet

Linea Mainnet

Base Mainnet

Base Sepolia

Polygon PoS Mainnet

Mantle Mainnet

Morph Mainnet

Introduction

The Optimistic Machine Learning (OPML) framework connects off-chain machine learning computations with smart contracts. Results are provably correct and can be disputed on-chain to ensure trust and accountability, enabling efficient AI-driven decision-making in decentralized applications. OPML is a core component of the AI Oracle, playing a critical role in validating inference results.

Workflow

1. Initiate Request: The user contract sends an inference request to AI Oracle by calling the requestCallback function.

2. opML Request: AI Oracle creates an opML request based on the user contract request.

3. Event Emission: AI Oracle emits a requestCallback event collected by the opML node.

4. ML Inference: The opML node performs the AI model computation.

5. Result Submission: The opML node uploads the result on-chain.

6. Callback Execution: The result is dispatched to the user's smart contract via a callback function.

Challenge Process

1. Challenge Window: Begins after the result is submitted on-chain (step 5 above).

2. Verification: opML validators or any network participant can check the results and challenge the output if it is incorrect.

3. Result Update: If a challenge is successful, the incorrect result is updated on-chain.

4. Finality: After the challenge period, the result is finalized onchain and made immutable.

Using AI Oracle

As a user, to interact with AI Oracle, you can:

• Use the AI.ORA.IO frontend- view our video tutorial

• Directly via the Prompt contract on Etherscan

Via Etherscan

1. In Prompt contract's Read Contract section, call estimateFee with specified modelId.

1. In Prompt contract's Write Contract section, call calculateAIResult with fee (converted from wei to ether), prompt, and modelId.

1. In the AI Oracle contract, look for the new transaction that fulfills the request you sent.

1. In Prompt contract's Read Contract section, call getAIResult with previous modelId and prompt to view the AI inference result.

Retrieve Historical AI Inference

If you want to retrieve your historical AI inference (eg. AIGC image), you can find them on blockchain explorer:

1. Find your transaction for sending AI request, and enter the "Logs" tab. Example tx: https://etherscan.io/tx/0xfbfdb2efcee23197c5ea8487368a905385c84afdc465cab43bc1ad01da773404#eventlog

2. Access your requestId in "Logs" tab Example's case: "1928".

3. In OAO's smart contract, look for the Invoke Callback transaction with the same requestId. Normally, this transaction will be around the same time as the one in step 1. To filter transactions by date, click "Advanced Filter" and then the button near "Age".

4. Find the transaction for AI inference result, and enter the "Logs" tab. Example tx: https://etherscan.io/tx/0xfbfdb2efcee23197c5ea8487368a905385c84afdc465cab43bc1ad01da773404#eventlog

5. Access output data. Example's case: "QmecBGR7dD7pRtY48FEKoeLVsmBTLwvdicWRkX9xz2NVvC" which is the IPFS hash that can be accessed with IPFS gateway)

ORA's AI Oracle is a verifiable oracle protocol that allows developers to create smart contracts and applications that ingest verifiable machine learning (ML) inferences for uses on the blockchain.

AI Oracle is powered by Optimistic Machine Learning (opML). It enables a verifiable, transparent, and decentralized way to integrate advanced ML models like LLaMA 3 and Stable Diffusion into smart contracts.

It offers:

• Scalability: Can run any ML model onchain without prohibitive overhead.

• Efficiency: Reduces computational time and cost when compared to zkML.

• Practicality: Can easily be integrated into applications by using existing blockchain infrastructure.

• Verifiability: Leverages fraud proofs to provide applications and users with assured computational integrity.

Components

AI Oracle comprises a set of smart contracts and off-chain components:

1. opML Contract: Handles fraud proofs and challenges to ensure on-chain verifiability of ML computations.

2. AIOracle Contract: Connects the opML node with on-chain callers to process ML requests and integrates various ML models.

3. User Contract: Customizable contracts that initiate AI inference requests and receive results from directly from AI Oracle.

4. ORA Nodes: Machines that run the Tora Client that interact with the ORA network. Nodes currently perform two functions: submit or validate inference results.

To showcase how AI Oracle can enhance consumer-facing products, we introduce Fortune Teller.

About Fortune Teller

Fortune Teller leverages the NestedInference smart contract, as discussed in the Advanced Usages of AI Oracle. This application aims to onboard new users to Web3 and demonstrate the capabilities of AI Oracle.

Application Flow

1. User Interaction: The application prompts users to answer specific questions posed by the Magic Wizard.

2. Fortune Telling: The Wizard casts its spells by requesting inferences from AI Oracle.

3. Response Generation: The Llama3 model generates a textual response, which is then used as a prompt for image creation via StableDiffusion.

4. NFT Minting: Users can mint AI-generated NFTs if they are satisfied with their fortune image.

The image on the right represents user's fortune result generated by AI Oracle:

• Text: Generated by Llama3 (onchain tx)

• Image: Generated by Stable Diffusion v3 (onchain tx)

Benefits of using AI Oracle in Various Applications

• Objective AI-Generated Insights: AI provides neutral and unbiased outputs, ensuring a fair experience across diverse use cases.

• Immutable Onchain Data: Information generated is securely stored on the blockchain, making it tamper-proof and easily verifiable.

• Transparent Data Generation: Utilizing opML, the entire generation process is transparent, fostering trust in the system across different applications.

Build your own AI dapps!

💡 If you're interested in creating your own frame, check out our AI Oracle Frame Template repository. This template can help you bootstrap your application, but you'll need to modify it based on your specific use case.

We encourage you to develop your own applications. Explore ideas in the awesome-ora repository, whether it’s a Farcaster frame, a Telegram mini app, or any other client application that interacts with AI Oracle.

In this tutorial, we’ll explore advanced techniques for interacting with the AI Oracle. Specifically, we’ll dive into topics like , , and Data Availability (DA) Options, giving you a deeper understanding of these features and how to leverage them effectively.

1. Nested Inference

A user can perform nested inference by initiating a second inference based on the result of the first inference within a smart contract. This action can be completed atomically and is not restricted to a two-step function.

Nested Inference Use Cases

Some of the use cases for a nested inference call include:

• generating a prompt with LLM for AIGC (AI Generated Content) NFT

• extracting data from a data set, then generate visual data with different models

• adding transcript to a video, then translate it to different languages with different models

For demo purposes we built a that uses ORA's AI Oracle.

Implementing Nested Inference

The idea of Nested Inference contract is to execute multiple inference requests in 1 transaction. We'll modify contract to support nested inference request. In our example, it will call Llama3 model first, then use inference result as the prompt to another request to StableDiffusion model.

The main goal of this tutorial is to understand what changes we need to make to Prompt contract in order to implement logic for various use cases.

Implementation Steps

1. modify CalculateAIResult method to support multiple requests

2. modify aiOracleCallback with the logic to handle second inference request

💡 When estimating gas cost for the callback, we should take both models into the consideration.

CalculateAIResult

As we now have additional function parameter for second model id. Not that we encode and forward model2Id as a callback data in aiOracle.requestCallback call.

aiOracleCallback

The main change here is the within "if" block. If the callback data (model2Id) is returned, we want to execute second inference request to the AI Oracle.

Output from the first inference call, will be passed to second one. This allows for interesting use cases, where you can combine text-to-text (eg. Llama3) and text-to-image (eg. Stable-Diffusion) models.

If nested inference call is not successful the whole function will revert.

💡 When interacting with the contract from the client side, we need to pass cumulative fee (for both models), then for each inference call we need to pass part of that cumulative fee. This is why we are calling estimateFee for model2Id.

Interaction with Contract

This is an example of contract interaction from Foundry testing environment. Note that we're estimating fee for both models and passing cumulative amount during the function call (we're passing slightly more to ensure that the call will execute if the gas price changes).

Conclusion

2. Batch Inference

The Batch Inference feature enables sending multiple inference requests within a single transaction, reducing costs by saving on network fees and improving the user experience. This bulk processing allows for more efficient handling of requests and results, making it easier to manage the state of multiple queries simultaneously.

Some of the use cases might be:

• AIGC NFT marketplace - creating a whole AIGC NFT collection with just one transaction, instead of creating those with many transactions

• Apps that need to handle requests simultaneously - Good example would be a recommendation system or chatbot with high TPS

Pricing

Callback transaction fee is needed for AI Oracle to submit callback transaction. It's calculated by multiplying current gas price with callback gas limit for invoked model (gasPrice * callbackGasLimit).

Request transaction fee is regular blockchain fee needed to request inference by invoking aiOracle.requestCallback method.

Total fee is calculated as sum of Model fee, Callback transaction fee and Request transaction fee.

Implementing Batch Inference

In order to initiate batch inference request, we will interact with requestBatchInference method. This method takes additional batchSize parameter, which specifies the amount of requests to the AI Oracle.

Prompt Format

Input for batch inference should be structured as a string representing an array of prompt and seed values.

Prompt - string value that is mandatory in order to prompt AI Oracle.

Seed – an optional numeric value that, when used, allows you to generate slightly varied responses for the same prompt.

Output Format

Result of Batch inference call is a dot separated list of inference results.

Example

This is the prompt for interacting with Stable Diffusion model:

Result is dot separated list of ipfs CIDs:

Interaction with Batch Inference

To test it, you need to:

1. create new javascript file

2. copy the script and add env variables

3. create and deploy prompt contract that supports batch inference

4. add values to batchInference_abi and batchInference_address

Prompt examples

Conclusion