opML (Optimistic Machine Learning), invented and developed by ORA, introduces a groundbreaking approach to integrating machine learning with blockchain technology. By leveraging similar principle of optimistic rollups, opML ensures the validity of computations in a decentralized manner. This framework enhances transparency and fosters trust in machine learning inference by allowing for onchain verification of AI computation.
OpML is comprised from the following key components:
Fraud Proof Virtual Machine (Off-chain VM): A robust off-chain engine responsible for executing machine learning inference. This component executes machine learning inference, generating new VM states as outputs. When discrepancies occur, manifested as different VM states, the MIPS VM employs a bisection method to pinpoint the exact step, or instruction, where the divergence begins.
opML Smart Contracts (On-chain VM) : Utilized for the verification of computational results, ensuring the accuracy of the off-chain computation. These contracts allow the execution of a single MIPS instruction, enabling the on-chain environment to verify specific steps in the computation process. This capability is vital for resolving disputes and ensuring the integrity of the off-chain computation.
Fraud Proofs: In the event of a dispute, fraud proofs generated by the verifier serve as conclusive evidence, illustrating the discrepancy in computation and facilitating the resolution process through the opML smart contracts.
Verification game is the process where two or more parties are assumed to execute the same program. Then, the parties can challenge each other with a pinpoint style to locate the disputable step. This step is sent to the smart contract for the verification.
For the system to work as intended it's important to ensure:
Deterministic ML execution
opML ensures consistent ML execution by using fixed-point arithmetic and software-based floating-points, eliminating randomness and achieving deterministic outcomes with a state transition function.
Separate Execution from Proving
opML utilizes a dual-compilation method: one for optimized native execution and another for fraud-proof VM instructions for secure verification. This ensures both fast execution and reliable, machine-independent proof.
Efficiency of AI model inference in VM
The existing fraud proof systems that are widely adopted in the optimistic rollup systems need to cross-compile the whole computation into fraud proof VM instructions, which will result in inefficient execution and huge memory consumption. opML proposes a novel multi-phase protocol, which allows semi-native execution and lazy loading, which greatly speeds up the fraud proof process.
The requester first initiates an ML service task.
The server then finishes the ML service task and commits results on chain.
The verifier will validate the results. Suppose there exists a verifier who declares the results are wrong. It starts a verification game with verification game (bisection protocol) with the server and tries to disprove the claim by pinpointing one concrete erroneous step.
Finally, arbitration about a single step will be conducted on smart contract.
Represents an extension of single-phase verification game, which allows for a better utilization of computing resources.
Single phase verification game cross-compiles the whole ML inference code into the Fraud Proof VM instructions. This method is less efficient than the native execution (doesn't utilize the full potential of GPU/TPU acceleration and parallel processing). The Fraud Proof VM also has limited memory, which prevents loading of large models into the memory directly.
To address the issues above, multi-phase verification game introduces the following properties:
Semi-Native Execution With the multi-phase design, we only need to conduct the computation in the VM only in the final phase, resembling the single-phase protocol. For other phases, we have the flexibility to perform computations that lead to state transitions in the native environment, leveraging the capabilities of parallel processing in CPU, GPU, or even TPU. By reducing the reliance on the VM, we significantly minimize overhead, resulting in a remarkable enhancement in the execution performance of opML, almost akin to that of the native environment
Lazy Loading Design To optimize the memory usage and performance of the fraud proof VM, we implement a lazy loading technique. This means that we do not load all the data into the VM memory at once, but only the keys that identify each data item. When the VM needs to access a specific data item, it uses the key to fetch it from the external source and load it into the memory. Once the data item is no longer needed, it is swapped out of the memory to free up space for other data items. This way, we can handle large amounts of data without exceeding the memory capacity or compromising the efficiency of the VM.
Detailed explanation of opML can be found in our research paper.
Check out ORA's open-source implementation repository.
opp/ai (Optimistic Privacy-Preserving AI), invented by ORA, represents an endgame onchain AI framework and an innovative approach to addressing the challenges of privacy and computational efficiency in blockchain-based machine learning systems. Opp/ai integrates Zero-Knowledge Machine Learning (zkML) for privacy with Optimistic Machine Learning (opML) for efficiency, creating a hybrid model tailored for onchain AI.
Opp/ai, as the latest fusion of zkML and opML, can include any zkML approach. It means that advances in zkML will be directly reflected in opp/ai.
Opp/ai can be utilized to conceal the fine-tuning weights of models where the majority of the weights are already publicly available. This is particularly relevant for open-source models that have been fine-tuned for specialized tasks. For instance, the LoRA weights in the attention layers of the Stable Diffusion model can be protected using opp/ai framework.
This capability is crucial for preserving the proprietary enhancements made to publicly shared models, ensuring that while the base model remains accessible, the unique adaptations that provide competitive advantage remain confidential.
Individual voice tuning in text-to-voice models: Text-to-voice service providers may offer personalized voice models that are tailored to the individual's voice characteristics. These personalized models are sensitive and contain valuable data. The opp/ai framework can ensure that the personalized voice model's parameters remain confidential while still offering the service to end-users verifiably.
Financial sector: Trading algorithms are developed to predict market movements and execute trades automatically. These algorithms are highly valuable and contain sensitive strategies that firms wish to protect. A financial institution could use the opp/ai framework to conceal the weights of a model that has been specifically tuned to its trading strategy.
Gaming industry: AI models are used to create challenging and engaging non-player characters (NPCs). Game developers may fine-tune these models to create unique behaviors or strategies that are specific to their game. By using the opp/ai framework, developers can hide the fine-tuned weights that contribute to the NPCs' competitive edge, preventing other developers from copying these features while still providing an immersive gaming experience.
Research paper on opp/ai.
To establish a verifiable and decentralized oracle network, it's critical to ensure the computation validity of results on the blockchain. This process involves a proof system that ensures the computation is reliable and truthful. By doing so, we can enhance the integrity and trustworthiness of decentralized applications that rely on any-size compute, including AI inference.
Several technologies invented and developed by ORA have emerged to facilitate the verifiable computation including AI inference on the blockchain. These innovations include Optimistic Machine Learning (opML), Keras2Circom (Zero-Knowledge Machine Learning, zkML), and Optimistic Privacy Preserving AI (opp/ai), with each representing a significant stride towards integrating verifiable proofs into the blockchain.
https://github.com/ora-io/keras2circom
ZkML is a proving framework that leverages Zero-Knowledge proofs to prove the validity of ML inference result on-chain. Due to its private nature it can protect confidential data and model parameters during training and inference, thus addressing privacy issues and reducing blockchain's computational load.
Keras2Circom, built by ORA, is the first advanced zkML framework that is battle-tested. From a recent benchmark by Ethereum Foundation ESP Grant Proposal [FY23-1290] on leading zkML frameworks, Keras2Circom and its underlying circomlib-ml are proven to be performant than other frameworks.
Besides being production-ready, circomlib-ml has rich ecosystem:
nova-ml by Ethereum Foundation
ORA leverages opML for Onchain AI Oracle because it’s the most feasible solution on the market for running any-size AI model onchain. The comparison between opML and zkML can be viewed from the following perspectives:
Proof system: opML uses fraud proofs, while zkML uses zk proofs.
Security: opML uses crypto-economic based security, while zkML uses cryptography based security.
Finality: We can define the finalized point of zkML and opML as follows:
zkML: Zero-knowledge proof of ML inference is generated (and verified).
opML: Challenge period of ML inference is passed. With additional mechanisms, faster finality can be achieved in much shorter time than the challenge period.
Opp/AI combines both opML and zkML approaches to achieve scalability and privacy. It preserves privacy while being more efficient than zkML.
Compared to pure zkML, opp/ai has much better performance with the same privacy feature.
