sRFC 21 - Nested Account Resolution

sRFC
, , ,
1

Code: GitHub - ngundotra/srfc-21-nested-account-resolution

SRFC 21 - Nested Account Resolution

Summary

This sRFC introduces a standard protocol to enable on-chain and off-chain derivation of accounts needed to complete a transaction, allowing the definition of account-free program interfaces.

Motivation

Composing programs on Solana is hard because it is difficult to imagine being able to swap out any program for any other program. So program developers only call out to one program at a time. This process leads to hard-coded composition paths.

This is one of the most frustrating aspects of Solana programming. This specification aims to solve this by providing a base for community members to decide on program interfaces.

Background

Defining program interfaces requires specifying accounts and instruction data. Interfaces with a strict list of accounts will restrict program implementation, because programs may be required to put all their information into a few accounts that fit the interface description. Interfaces defined with a strict list of accounts restricts innovation in program development, and fails to take advantage of the SVM’s composability.

We want to enable interfaces to specify required accounts without restricting additional account usage. We believe this will allow exciting developments of innovation with program interfaces.

Two problems with this

  1. How can programs specify additional accounts they need for a specific instruction?
  2. If a program is CPI-ing into a unknown program that requests additional accounts, how can it expose those requested accounts, and then reliably pass such accounts to the unknown program?

Specification: Nested Account Resolution

We propose to solve these two problems by allowing programs to request additional accounts for an instruction with an additional instruction that requests accounts using return data.

Off-chain clients can simulate the additional instruction, parse the return data, and then construct a valid transaction for the program’s instruction using the requested additional accounts.

As of right now, this specification requires 3 things:

  1. Program must have an Anchor IDL deployed on-chain
  2. Instructions that request additional accounts must have 8-byte instruction discriminator derived from their name and a corresponding entry in the program’s anchor IDL.
  3. The additional instruction that requests additional accounts must also have an 8-byte instruction discriminator derived from the target instructions name, and a corresponding entry in the program’s anchor IDL.

These additional instructions are called aar instructions, e.g. additionalAccountsRequest, or additionalAccountResolution.

Let’s go through how these are meant to be used before diving in any deeper. Our examples will focus on programs built with the anchor framework, since it meant to improve program composability and transparency on Solana.

Additional accounts request instructions, when implemented look like the following

use anchor_lang::prelude::*;
use additional_accounts_request::AdditionalAccounts;
use solana_program::program::set_return_data;

#[program]
pub mod my_program {
  use super::*;

  pub fn transfer(ctx: Context<Transfer>, args: TransferArgs) -> Result<()> {
    ...
  }

  pub fn aar_transfer(ctx: Context<TransferReadonly>, args: TransferArgs) -> Result<()> {
    let mut requested_accounts = AdditionalAccounts::new();
    ...
    requested_accounts.add_account(&pubkey, is_writable)?;
    ...
    set_return_data(bytemuck::bytes_of(&additional_accounts));

    Ok(())
  }
}

Off-chain clients can compose a valid TransactionInstruction for transfer by simulating aar_transfer with TransferReadonly’s accounts successively until requested_accounts.has_more is false. Each successive simulation of aar_transfer must include the accounts requested in the previous simulation. This is to allow for account derivation logic that requires account data, which is only available in simulation when the account is provided.

The typescript pseudo code looks like:

// Create base AAR instruction
let instruction: TransactionInstruction = await myProgram
  .methods
  .aarTransfer(args)
  .accounts({
    ...
}).instruction();

let originalKeys: AccountMeta[] = instruction.keys
let additionalAccounts: AccountMeta[] = [];

while (hasMore) {
    // Add previously requested accounts to instruction
    instruction.keys = originalKeys.concat(additionalAccounts.flat());

    // Simulate instruction and get return data
    let returnData = await simulateTransactionInstruction(
      connection,
      instruction,
    );

    // Deserialize return data into the Additional Accounts
    let additionalAccountsRequest = AdditionalAccountsRequest.fromBuffer(returnData);

    // Store requested accounts
    hasMore = additionalAccountsRequest.hasMore;
    additionalAccounts = additionalAccounts.concat(additionalAccountsRequest.accounts);
}

Here’s the structure of the return data to be set by any additional accounts request method.

pub const MAX_ACCOUNTS: usize = 30;
#[zero_copy]
#[derive(Debug, AnchorDeserialize, AnchorSerialize)]
pub struct AdditionalAccounts {
    pub protocol_version: u8,
    pub has_more: u8,
    pub _padding_1: [u8; 2],
    pub num_accounts: u32,
    pub accounts: [Pubkey; MAX_ACCOUNTS],
    pub writable_bits: [u8; MAX_ACCOUNTS],
    pub _padding_2: [u8; 26],
}

We chose AdditionalAccounts to have the maximum number of account meta information possible, while still keeping the total size under 1024 bytes (maximum amount of Solana return data).

An additional requirement was keeping the struct byte-aligned so that the entire struct could be used with zero copy to excess heap usage.

Passing exposed accounts requested by CPIs

The hard part here is knowing which of the requested accounts should be used for a given CPI, especially if you have multiple CPIs to unknown programs (e.g. swapping ownership of assets between two programs).

We propose a low compute solution that uses an “account delimiter” to group account segments together of variable length. The account delimiter for a program is a PDA with seeds &["DELIMITER".as_ref()].

An example swap implementation is given here:

pub fn swap(ctx: Context<Swap>) -> Result<()> {
  ...

  // Track consumed accounts
  let mut delimiter_idx = call(
      ix_name_0,
      cpi_ctx_0,
      args_0,
      get_delimiter(&crate::id()),
      0
  )?;
  
  // Filter out consumed accounts
  delimiter_idx = call(
      ix_name_1,
      cpi_ctx_1,
      args_1,
      get_delimiter(&crate::id()),
      delimiter_idx,
  )?;

  Ok(())
}

Exposing accounts requested by CPI

pub fn preflight_swap(ctx: Context<SwapReadonly>) -> Result<()> {
  # find number of account delimiters
  let mut latest_delimiter_idx = 0;
  let mut stage = 0;
  ctx.remaining_accounts
      .iter()
      .enumerate()
      .for_each(|(i, acc)| {
          if acc.key() == delimiter {
              stage += 1;
              latest_delimiter_idx = i;
          }
      });

  # Based off # of delimiters, I can derive how many CPIs have finished requested account
  match stage {
    0 => {
      # CPI to `aar` instruction for unknown program A
      # pass all undelimited accounts to program A
      if done { 
        addtional_accounts.add_account(&get_delimiter(&crate::id()), false)?; 
      }
    }
    1 => {
      # CPI to `aar` instruction for unknown program B
      # pass all undelimited accounts to program B
    }
    _ => {
     msg!("Too many delimiters passed");
     Err(ProgramError::InvalidInstructionData.into())
   }
  }
}

Limitations

  1. This spec does not support requesting additional signer accounts.

Allowing unknown programs to request the signature of accounts could be a security vulnerability, and would be technically challenging to implement. So supporting the signer accounts must come in the form of a new specification.

  1. Resolving additional accounts can be slow

Successively simulating the aar instruction with results of the last call can be quite slow, and there is no maximum number of iterations defined by this specification. There is no current guide or set of heuristics on how to cache requested additional accounts yet either. This means that applications may see increased RPC calls for simulateTransaction and increased latency when showing end-users transactions.

Future Work

  1. Anchor macros to build aar instructions

It is possible to design macros that build aar instruction at compile-time entirely from the anchor instruction struct and a list of CPI callsites. We hope that if the community adopts this sRFC, additional developer tooling will be made available here.

  1. Support for state compression

Once this specification proves valuable, it would be quite possible to support state compression, since the proofs to a ConcurrentMerkleTree only require a list of accounts. However, doing so would require referencing off-chain indexers. This seems best suited for different spec and protocol version of AdditionalAccountsRequest.

  1. Faster account resolution

It seems quite possible to write a thin wrapper around simulateTransaction that uses Geyser to stream account updates to Solana Banks Test, so that transaction simulations are faster and the results can be cached for quicker lookup. This would require a lot of work, but this would probably greatly increase adoption speed.

Code

Library with helper functions for implementing this spec is available at: docs.rs/additional-accounts-request

Integration tests for the library are available at GitHub - ngundotra/srfc-21-nested-account-resolution: Nested account resolution library, PoC, examples, and documentation with yarn install && yarn test.

4 Likes