In this tutorial, we’ll explore advanced techniques for interacting with the AI Oracle. Specifically, we’ll dive into topics like Nested Inference, Batch Inference, and Data Availability (DA) Options, giving you a deeper understanding of these features and how to leverage them effectively.
A user can perform nested inference by initiating a second inference based on the result of the first inference within a smart contract. This action can be completed atomically and is not restricted to a two-step function.
Some of the use cases for a nested inference call include:
generating a prompt with LLM for AIGC (AI Generated Content) NFT
extracting data from a data set, then generate visual data with different models
adding transcript to a video, then translate it to different languages with different models
For demo purposes we built a farcaster frame that uses ORA's AI Oracle.
The idea of Nested Inference contract is to execute multiple inference requests in 1 transaction. We'll modify Prompt contract to support nested inference request. In our example, it will call Llama3 model first, then use inference result as the prompt to another request to StableDiffusion model.
The main goal of this tutorial is to understand what changes we need to make to Prompt contract in order to implement logic for various use cases.
modify CalculateAIResult
method to support multiple requests
modify aiOracleCallback
with the logic to handle second inference request
💡 When estimating gas cost for the callback, we should take both models into the consideration.
As we now have additional function parameter for second model id. Not that we encode and forward model2Id
as a callback data in aiOracle.requestCallback
call.
The main change here is the within "if" block. If the callback data (model2Id
) is returned, we want to execute second inference request to the AI Oracle.
Output from the first inference call, will be passed to second one. This allows for interesting use cases, where you can combine text-to-text (eg. Llama3) and text-to-image (eg. Stable-Diffusion) models.
If nested inference call is not successful the whole function will revert.
💡 When interacting with the contract from the client side, we need to pass cumulative fee (for both models), then for each inference call we need to pass part of that cumulative fee. This is why we are calling estimateFee
for model2Id
.
This is an example of contract interaction from Foundry testing environment. Note that we're estimating fee for both models and passing cumulative amount during the function call (we're passing slightly more to ensure that the call will execute if the gas price changes).
You can also check the full implementation of PromptNestedInference.
The Batch Inference feature enables sending multiple inference requests within a single transaction, reducing costs by saving on network fees and improving the user experience. This bulk processing allows for more efficient handling of requests and results, making it easier to manage the state of multiple queries simultaneously.
Some of the use cases might be:
AIGC NFT marketplace - creating a whole AIGC NFT collection with just one transaction, instead of creating those with many transactions
Apps that need to handle requests simultaneously - Good example would be a recommendation system or chatbot with high TPS
Model fee required for batch inference request is calculated by multiplying single model fee with batch size (batchSize * model.fee
). This fee covers the operational costs of running AI models, with a portion contributing to protocol revenue. For details on the required fees for each model, visit the References page.
Callback transaction fee is needed for AI Oracle to submit callback transaction. It's calculated by multiplying current gas price with callback gas limit for invoked model (gasPrice * callbackGasLimit
).
Request transaction fee is regular blockchain fee needed to request inference by invoking aiOracle.requestCallback
method.
Total fee is calculated as sum of Model fee, Callback transaction fee and Request transaction fee.
In order to initiate batch inference request, we will interact with requestBatchInference
method. This method takes additional batchSize parameter, which specifies the amount of requests to the AI Oracle.
Note that we'll need to pass more gas to cover AI Oracle callback execution, depending on the batchSize (check out Pricing). For this purpose we can implement estimateFeeBatch
function in our Prompt contract. This method will interact with estimateFeeBatch
method from AIOracle.sol.
Input for batch inference should be structured as a string representing an array of prompt and seed values.
Prompt - string value that is mandatory in order to prompt AI Oracle.
Seed – an optional numeric value that, when used, allows you to generate slightly varied responses for the same prompt.
Result of Batch inference call is a dot separated list of inference results.
This is the prompt for interacting with Stable Diffusion model:
Result is dot separated list of ipfs CIDs:
When performing batch inference with the AI Oracle, ensure the prompt is following the standard format. Below is a simple script for interacting with batch inference:
To test it, you need to:
create new javascript file
copy the script and add env variables
create and deploy prompt contract that supports batch inference
add values to batchInference_abi and batchInference_address
Prompt examples
That's it! With few simple changes to the Prompt contract we utilised batch inference feature. This allowed us to get multiple responses with only one transaction.
Build your dApp using ORA's AI Oracle
This tutorial will help you understand the structure of the AI Oracle, guide you through the process of building a simple Prompt contract that interacts with the ORA network.
If you prefer a video version of the tutorial, check it here.
Final version of the code can be found here.
Setup the development environment
Understand the project setup and template repository structure
Learn how to interact with the AI Oracle and build an AI powered smart contract
To follow this tutorial you need to have Foundry and git installed.
Clone template repository and install submodules
Move into the cloned repository
Copy .env.example, rename it to .env. We will need these env variables later for the deployment and testing. You can leave them empty for now.
Install foundry dependencies
At the beginning we need to import several dependencies which our smart contract will use.
IAIOracle - interface that defines a requestCallback
method that needs to be implemented in the Prompt contract
AIOracleCallbackReceiver - an abstract contract that contains an instance of AIOracle and implements a callback method that needs to be overridden in the Prompt contract
We'll start by implementing the constructor, which accepts the address of the deployed AIOracle contract.
Now let’s define a method that will interact with the AI Oracle. This method requires 2 parameters, id of the model and input prompt data. It also needs to be payable, because a user needs to pass the fee for the callback execution.
In the code above we do the following:
Convert input to bytes
Call the requestCallback function with the following parameters:
modelId: ID of the AI model in use.
input: User-provided prompt.
callbackAddress: The address of the contract that will receive AI Oracle's callback.
callbackGasLimit[modelId]: Maximum amount of that that can be spent on the callback, yet to be defined.
callbackData: Callback data that is used in the callback.
Next step is to define the mapping that keeps track of the callback gas limit for each model and set the initial values inside the constructor. We’ll also define a modifier so that only the contract owner can change these values.