Interaction with OAO - Tutorial covers step by step creation of simple Prompt contract that interacts with OAO. Integration into OAO - Tutorial covers the process of creating model for your AI model and integration into OAO.

In this tutorial we explain how to integrate your own AI model into Onchain AI Oracle (OAO). We will start by looking at mlgo repository and trying to understand what's happening there. At the end we will showcase how opML works, by running a simple dispute game script inside a docker container.

Learning Objectives

• Understand how to transform AI model and inference code in order to integrate it into Onchain AI Oracle (OAO).

• Execute a simple dispute game and understand the process of AI inference verification.

Prerequisites

• git installed

Setup

1. Clone mlgo repository

git clone git@github.com:OPML-Labs/mlgo.git

1. Navigate to cloned repository

cd mlgo

1. To install the required dependencies for your project, run the following command:

pip install -r requirements.txt

If there are some missing dependencies, make sure to install them in your Python environment.

Train the model using Pytorch

First we need to train a DNN model using Pytorch. The training part is shown in examples/mnist/trainning/mnist.ipynb.

After the training model is saved at examples/mnist/models/mnist/mnist-small.state_dict.

Convert model into ggml format

Ggml is a file format that consists of version number, followed by three components that define a large language model: the model's hyperparameters, its vocabulary, and its weights. Ggml allows for more efficient inference runs on CPU. We will now convert the Python model to ggml format by executing following steps:

1. Position to mnist folder

cd examples/mnist

1. Convert python model into ggml format

python3 convert-h5-to-ggml.py models/mnist/mnist-small.state_dict

In order to convert AI model written in Python to ggml format we are executing python script and providing a file that stores the model as a parameter to the script. The output is the binary file in ggml format. Note that the model is saved in big-endian, making it easy to process in the big-endian MIPS-32 VM.

Converting inference code to MIPS VM executable

Next step is to write inference code in go language. Then we will transform go binary into MIPS VM executable file.

Go supports compilation to MIPS. However, the generated executable is in ELF format. We'd like to get a pure sequence of MIPS instructions instead. To build a ML program in MIPS VM execute the following steps:

1. Navigate to the mnist_mips directory and build go inference code

cd ../mnist_mips && ./build.sh

Build script will compile go code and then run compile.py script that will transform compiled go code to the MIPS VM executable file.

Running the dispute game

Now that we compiled our AI model and inference code into MIPS VM executable code.

We can test the dispute game process. We will use a bash script from opml repository to showcase the whole verification flow.

For this part of the tutorial we will use Docker, so make sure to have it installed.

Let's first check the content of the Dockerfile that we are using:

1. First we need to specify the operating system that runs inside our container. In this case we're using ubuntu:22.04.

# How to run instructions:
# 1. Generate ssh command: ssh-keygen -t rsa -b 4096 -C "your_email@example.com"
#    - Save the key in local repo where Dockerfile is placed as id_rsa
#    - Add the public key to the GitHub account
# 2. Build docker image: docker build -t ubuntu-opml-dev .
# 3. Run the hardhat: docker run -it --rm --name ubuntu-opml-dev-container ubuntu-opml-dev bash -c "npx hardhat node"
# 4. Run the challange script on the same container: docker exec -it ubuntu-opml-dev-container bash -c "./demo/challenge_simple.sh"


# Use an official Ubuntu as a parent image
FROM ubuntu:22.04

1. Then we need to install all the necessary dependencies in order to run dispute game script.

# Set environment variables to non-interactive to avoid prompts during package installations
ENV DEBIAN_FRONTEND=noninteractive

# Update the package list and install dependencies
RUN apt-get update && apt-get install -y \
    build-essential \
    cmake \
    git \
    golang \
    wget \
    curl \
    python3 \
    python3-pip \
    python3-venv \
    unzip \
    file \
    openssh-client \
    && apt-get clean \
    && rm -rf /var/lib/apt/lists/*

# Install Node.js and npm
RUN curl -fsSL https://deb.nodesource.com/setup_18.x | bash - && \
    apt-get install -y nodejs

1. Then we configure ssh keys, so that docker container can clone all the required repositories.

# Copy SSH keys into the container
COPY id_rsa /root/.ssh/id_rsa
RUN chmod 600 /root/.ssh/id_rsa
# Configure SSH to skip host key verification
RUN echo "Host *\n\tStrictHostKeyChecking no\n" >> /root/.ssh/config

1. Position to the root directory inside docker container and clone opml repository along with its submodules.

# Set the working directory
WORKDIR /root

# Clone the OPML repository
RUN git clone git@github.com:ora-io/opml.git --recursive
WORKDIR /root/opml

1. Lastly, we tell docker to build executables and run the challenge script.

# Build the OPML project
RUN make build

# Change permission for the challenge script
RUN chmod +x demo/challenge_simple.sh

# Default command
CMD ["bash"]

Create docker container and run the script

1. In order to successfully clone opml repository, you need to generate a new ssh key and add it to your Github account. Once it's generated, save the key in the local repository where Dockerfile is placed as id_rsa. Then add the public key to your GitHub account.

   Copy

   ssh-keygen -t rsa -b 4096 -C "your_email@example.com"

2. Build the docker image

   Copy

   docker build -t ubuntu-opml-dev .

3. Run the local Ethereum node

   Copy

   docker run -it --rm --name ubuntu-opml-dev-container ubuntu-opml-dev bash -c "npx hardhat node"

4. In another terminal run the challenge script

   Copy

   docker exec -it ubuntu-opml-dev-container bash -c "./demo/challenge_simple.sh"

After executing the steps above you should be able to see interactive challenge process in the console.

Script first deploys necessary contracts to the local node. Proposer opML node executes AI inference and then the challenger nodes can dispute it, if they think that the result is not valid. Challenger and proposer are interacting in order to find the differing step between their computations. Once the dispute step is found it's sent to the smart contract for the arbitration. If the challenge is successful the proposer node gets slashed.

Conclusion

In this tutorial we achieved the following:

• converted our AI model from Python to ggml format

• compiled AI inference code written in go to MIPS VM executable format

• run the dispute game inside a docker container and understood the opML verification process

Next steps

In order to use your AI model onchain, you need to run your own opML nodes, then this AI model will be able to integrated into OAO. Try to reproduce this tutorial with your own model.

This tutorial will help you understand the structure of Onchain AI Oracle (OAO), guide you through the process of building a simple Prompt contract that interacts with OAO. We will implement the contract step by step. At the end we will deploy the contract to the blockchain network and interact with it.

If you prefer a video version of the tutorial, check it here.

Final version of the code can be found here.

Learning Objectives

• Setup development environment

• Understand the project setup and template repository structure

• Learn how to interact with the OAO and build an AI powered smart contract

Prerequisites

To follow this tutorial you need to have Foundry and git installed.

Setup

1. Clone template repository and install submodules

git clone -b OAO_interaction_tutorial git@github.com:ora-io/Interaction_With_OAO_Template.git --recursive

1. Move into the cloned repository

cd Interaction_With_OAO_Template

1. Copy .env.example, rename it to .env. We will need these env variables later for the deployment and testing. You can leave them empty for now.

cp .env.example .env

1. Install foundry dependencies

forge install

Creating Prompt contract

1. Import dependencies

At the beginning we need to import several dependencies which our smart contract will use.

import "OAO/contracts/interfaces/IAIOracle.sol";
import "OAO/contracts/AIOracleCallbackReceiver.sol";

IAIOracle - interface that defines a requestCallback method that needs to be implemented in the Prompt contract AIOracleCallbackReceiver - an abstract contract that contains an instance of AIOracle and implements a callback method that needs to be overridden in the Prompt contract

2. Write the code

We'll start by implementing the constructor, which accepts the address of the deployed AIOracle contract.

constructor(IAIOracle _aiOracle) AIOracleCallbackReceiver(_aiOracle){}

Now let’s define a method that will interact with the OAO. This method takes 2 parameters, id of the model and input prompt data. It also needs to be payable, because a user needs to pass the fee for the callback execution.

function calculateAIResult(uint256 modelId, string calldata prompt) payable external {
    bytes memory input = bytes(prompt);
    bytes memory callbackData = bytes("");
    address callbackAddress = address(this);
    uint256 requestId = aiOracle.requestCallback{value: msg.value}(
        modelId, input, callbackAddress, callbackGasLimit[modelId], callbackData
    );
}

In the code above we do the following:

1. Convert input to bytes

2. Call the requestCallback function with the following parameters:

   • modelId: ID of the AI model in use.

   • input: User-provided prompt.

   • callbackAddress: The address of the contract that will receive OAO's callback.

   • callbackGasLimit[modelId]: Maximum amount of that that can be spent on the callback, yet to be defined.

   • callbackData: Callback data that is used in the callback.

Next step is to define the mapping that keeps track of the callback gas limit for each model and set the initial values inside the constructor. We’ll also define a modifier so that only the contract owner can change these values.

address owner;

modifier onlyOwner() {
    require(msg.sender == owner, "Only owner");
    _;
}

mapping(uint256 => uint64) public callbackGasLimit;

function setCallbackGasLimit(uint256 modelId, uint64 gasLimit) external onlyOwner {
    callbackGasLimit[modelId] = gasLimit;
}

constructor(IAIOracle _aiOracle) AIOracleCallbackReceiver(_aiOracle) {
    owner = msg.sender;
    callbackGasLimit[50] = 500_000; // Stable-Diffusion
    callbackGasLimit[11] = 5_000_000; // Llama
}

We want to store all the requests that happened, so we create a data structure for the request data and the mapping between requestId and the request data.

event promptRequest(
    uint256 requestId,
    address sender, 
    uint256 modelId,
    string prompt
);

struct AIOracleRequest {
    address sender;
    uint256 modelId;
    bytes input;
    bytes output;
}

mapping(uint256 => AIOracleRequest) public requests;

function calculateAIResult(uint256 modelId, string calldata prompt) payable external {
    bytes memory input = bytes(prompt);
    bytes memory callbackData = bytes("");
    address callbackAddress = address(this);
    uint256 requestId = aiOracle.requestCallback{value: msg.value}(
        modelId, input, callbackAddress, callbackGasLimit[modelId], callbackData
    );
    AIOracleRequest storage request = requests[requestId];
    request.input = input;
    request.sender = msg.sender;
    request.modelId = modelId;
    emit promptRequest(requestId, msg.sender, modelId, prompt);
}

In the code snippet above we added prompt, sender and the modelId to the request and also emitted an event.

Now that we implemented a method for interaction with the OAO, let's define a callback that will be invoked by the OAO after the computation of the result.

event promptsUpdated(
    uint256 requestId,
    uint256 modelId,
    string input,
    string output,
    bytes callbackData
);

mapping(uint256 => mapping(string => string)) public prompts;

function getAIResult(uint256 modelId, string calldata prompt) external view returns (string memory) {
    return prompts[modelId][prompt];
}

function aiOracleCallback(uint256 requestId, bytes calldata output, bytes calldata callbackData) external override onlyAIOracleCallback() {
    AIOracleRequest storage request = requests[requestId];
    require(request.sender != address(0), "request does not exist");
    request.output = output;
    prompts[request.modelId][string(request.input)] = string(output);
    emit promptsUpdated(requestId, request.modelId, string(request.input), string(output), callbackData);
}

We've overridden the callback function from the AIOracleCallbackReceiver.sol. It's important to use the modifier, so that only OAO can callback into our contract.

Function flow:

1. First we check if the request with provided id exists. If it does we add the output value to the request.

2. Then we define prompts mapping that stores all the prompts and outputs for each model that we use.

3. At the end we emit an event that the prompt has been updated.

Notice that this function takes callbackData as the last parameter. This parameter can be used to execute arbitrary logic during the callback. It is passed during requestCallback call. In our simple example, we left it empty. Finally let's add the method that will estimate the fee for the callback call.

function estimateFee(uint256 modelId) public view returns (uint256) {
    return aiOracle.estimateFee(modelId, callbackGasLimit[modelId]);
}

With this we finished with the source code for our contract. The final version should look like this:

// SPDX-License-Identifier: MIT
pragma solidity ^0.8.13;

import "OAO/contracts/interfaces/IAIOracle.sol";
import "OAO/contracts/AIOracleCallbackReceiver.sol";

contract Prompt is AIOracleCallbackReceiver {
    
    event promptsUpdated(
        uint256 requestId,
        uint256 modelId,
        string input,
        string output,
        bytes callbackData
    );

    event promptRequest(
        uint256 requestId,
        address sender, 
        uint256 modelId,
        string prompt
    );

    struct AIOracleRequest {
        address sender;
        uint256 modelId;
        bytes input;
        bytes output;
    }

    address owner;

    modifier onlyOwner() {
        require(msg.sender == owner, "Only owner");
        _;
    }

    mapping(uint256 => AIOracleRequest) public requests;

    mapping(uint256 => uint64) public callbackGasLimit;

    constructor(IAIOracle _aiOracle) AIOracleCallbackReceiver(_aiOracle) {
        owner = msg.sender;
        callbackGasLimit[50] = 500_000; // Stable-Diffusion
        callbackGasLimit[11] = 5_000_000; // Llama
    }

    function setCallbackGasLimit(uint256 modelId, uint64 gasLimit) external onlyOwner {
        callbackGasLimit[modelId] = gasLimit;
    }

    mapping(uint256 => mapping(string => string)) public prompts;

    function getAIResult(uint256 modelId, string calldata prompt) external view returns (string memory) {
        return prompts[modelId][prompt];
    }

    function aiOracleCallback(uint256 requestId, bytes calldata output, bytes calldata callbackData) external override onlyAIOracleCallback() {
        AIOracleRequest storage request = requests[requestId];
        require(request.sender != address(0), "request does not exist");
        request.output = output;
        prompts[request.modelId][string(request.input)] = string(output);
        emit promptsUpdated(requestId, request.modelId, string(request.input), string(output), callbackData);
    }

    function estimateFee(uint256 modelId) public view returns (uint256) {
        return aiOracle.estimateFee(modelId, callbackGasLimit[modelId]);
    }

    function calculateAIResult(uint256 modelId, string calldata prompt) payable external {
        bytes memory input = bytes(prompt);
        bytes memory callbackData = bytes("");
        address callbackAddress = address(this);
        uint256 requestId = aiOracle.requestCallback{value: msg.value}(
            modelId, input, callbackAddress, callbackGasLimit[modelId], callbackData
        );
        AIOracleRequest storage request = requests[requestId];
        request.input = input;
        request.sender = msg.sender;
        request.modelId = modelId;
        emit promptRequest(requestId, msg.sender, modelId, prompt);
    }
}

Deployment and interaction with the contract

Foundry version

1. Add your PRIVATE_KEY, RPC_URL and ETHERSCAN_KEY to .env file. Then source variables in the terminal.

   Copy

   source .env

2. Create a deployment script

   Go to Reference page and find the OAO_PROXY address for the network you want to deploy to.

   Then open script/Prompt.s.sol and add the following code:

   Copy

   // SPDX-License-Identifier: MIT
   pragma solidity ^0.8.13;
   
   import {Script} from "forge-std/Script.sol";
   import {Prompt} from "../src/Prompt.sol";
   import {IAIOracle} from "OAO/contracts/interfaces/IAIOracle.sol";
   
   contract PromptScript is Script {
       address OAO_PROXY;
   
       function setUp() public {
           OAO_PROXY = [OAO_PROXY_address_here];
       }
   
       function run() public {
           uint privateKey = vm.envUint("PRIVATE_KEY");
           vm.startBroadcast(privateKey);
           new Prompt(IAIOracle(OAO_PROXY));
           vm.stopBroadcast();
       }
   }

3. Run the deployment script

   Copy

   forge script script/Prompt.s.sol --rpc-url $RPC_URL --broadcast --verify --etherscan-api-key $ETHERSCAN_KEY

4. Once the contract is deployed and verified, you can interact with it. Go to blockchain explorer for your chosen network (eg. Etherscan), and paste the address of the deployed Prompt contract.

   Let's use Stable Diffusion model (id = 50).

   1. First call estimateFee method to calculate fee for the callback.
   2. Then request AI inference from OAO by calling calculateAIResult method. Pass the model id and the prompt for the image generation. Remember to provide estimated fee as a value for the transaction.
   3. After the transaction is executed, and the OAO calculates the result, you can check it by calling prompts method. Simply input model id and the prompt you used for image generation. In the case of Stable Diffusion, the output will be a CID (content identifier on ipfs). To check the image go to https://ipfs.io/ipfs/[Replace_your_CID].

Remix version

1. Install the browser wallet if you haven't already (eg. Metamask)

2. Open your solidity development environment. We'll use Remix IDE.

3. Copy the contract along with necessary dependencies to Remix.
4. Choose the solidity compiler version and compile the contract to the bytecode
5. Deploy the compiled bytecode Once we compiled our contract, we can deploy it.

   1. First go to the wallet and choose the blockchain network for the deployment.

   2. To deploy the Prompt contract we need to provide the address of already deployed AIOracle contract. You can find this address on the reference page. We are looking for OAO_PROXY address.

   3. Deploy the contract by signing the transaction in the wallet.
6. Once the contract is deployed, you can interact with it. Remix supports API for interaction, but you can also use blockchain explorers like Etherscan. Let's use Stable Diffusion model (id = 50).

Conclusion

In this tutorial we covered a step by step writing of a solidity smart contract that interacts with ORA's Onchain AI Oracle. Then we compiled and deployed our contract to the live network and interacted with it. In the next tutorial, we will extend the functionality of the Prompt contract to support AI generated NFT collections.