sRFC 00008: IDL Standard

sRFC 00008: IDL Standard

Summary

It’s no question that the introduction of an IDL alongside a Solana program was one of the biggest value-adds provided by Anchor.

With an IDL, indexers, explorers, and many other tools can gain insight into a deployed program that they otherwise couldn’t have gotten - especially deserialization of PDA data.

Right now, we continually run into problems when trying to develop tools or solutions that encounter two types of programs: one with an IDL published, and one without an IDL.
Immediately these projects have to essentially “skip” or “omit” programs that are deployed without an IDL, because their solution simply will not work without one.

We need a standard way to manage IDLs for programs across Solana, and it shouldn’t matter which framework or tools you use to build your program.

Questions


What should an IDL’s interface look like?

Two existing and popular tools for creating IDLs - Anchor and Shank - actually generate very similar IDLs.

Here’s part of a simple example:

{
  "version": "0.1.0",
  "name": "car_rental_service",
  "instructions": [

    ...

    {
      "name": "BookRental",
      "accounts": [
        {
          "name": "rentalAccount",
          "isMut": true,
          "isSigner": false,
          "desc": "The account that will represent the actual order for the rental"
        },
        {
          "name": "carAccount",
          "isMut": false,
          "isSigner": false,
          "desc": "The account representing the Car being rented in this order"
        },
        {
          "name": "payer",
          "isMut": true,
          "isSigner": false,
          "desc": "Fee payer"
        },
        {
          "name": "systemProgram",
          "isMut": false,
          "isSigner": false,
          "desc": "The System Program"
        }
      ],
      "args": [
        {
          "name": "bookRentalArgs",
          "type": {
            "defined": "BookRentalArgs"
          }
        }
      ],
      "discriminant": {
        "type": "u8",
        "value": 1
      }
    },

    ...

  ],
  "accounts": [

    ...

    {
      "name": "RentalOrder",
      "type": {
        "kind": "struct",
        "fields": [
          {
            "name": "car",
            "type": "publicKey"
          },
          {
            "name": "name",
            "type": "string"
          },
          {
            "name": "pickUpDate",
            "type": "string"
          },
          {
            "name": "returnDate",
            "type": "string"
          },
          {
            "name": "price",
            "type": "u64"
          },
          {
            "name": "status",
            "type": {
              "defined": "RentalOrderStatus"
            }
          }
        ]
      }
    }
  ],
  "types": [

    ...

    {
      "name": "RentalOrderStatus",
      "type": {
        "kind": "enum",
        "variants": [
          {
            "name": "Created"
          },
          {
            "name": "PickedUp"
          },
          {
            "name": "Returned"
          }
        ]
      }
    }
  ],
  "metadata": {
    "origin": "shank",
    "address": "8avNGHVXDwsELJaWMSoUZ44CirQd4zyU9Ez4ZmP4jNjZ",
    "binaryVersion": "0.0.12",
    "libVersion": "0.0.12"
  }
}

(Full IDL here)

Ultimately, we want to figure out if we can use this (or Anchor’s IDL) as the interface for all IDLs across Solana.


What Should be the Standard for Implementing the IDL Interface?

If we consider the IDL above - or something similar - to serve as our IDL interface, what should be the standard for implementing?

Can you just add whatever fields you want, as long as you still have the fields from the interface? For example:

"accounts": [

    ...

    {
      "name": "RentalOrder",
      "type": {
        "kind": "struct",
        "fields": [
          {
            "name": "car",
            "type": "publicKey"
          },
          {
            "name": "name",
            "type": "string"
          },
        ],

        "myCustomConfiguration": {
           "someConfig": 1,
           "someOtherConfig": "2",
        }

      }
    }
  ],

Considerations:

  • This would break any type-oriented IDL parsers by introducing new fields that otherwise weren’t part of the type schema
  • A lack of limits might inflate the size of IDLs unexpectedly

What crate/lib/types Should be Used?

We could do something like introduce a standard crate with the interface types as the basis for all IDLs on Solana.

We then could modify these types to allow for pluggable custom configurations, or some other means for easily implementing the interface for an IDL leveraging this crate.

A great candidate for this standard library of IDL types (interface) is Iron Forge’s Solana IDL - an open-source crate containing the IDL types from Shank compatible with serde.

It would be great to hear thoughts on:

  • Electing this crate as the IDL type interface
  • Modifying this crate to serve as an implementable interface, with customizable type configurations

Implementations & Ideas

Here’s a PR introduced by Noah from Solana Labs to introduce the idea of an IDL program within the Solana runtime.

Here’s a crate produced by Thorsten from Iron Forge to lay down the Rust types for creating an IDL leveraging serde.

Conclusion

In short, like any other interface or standard proposal, this is a migration that could be done in a way that hurts only once, but allows for easy integrations and added support in the future.
A quick recap on the questions proposed:

  • What should an IDL’s interface look like?
  • What should be the standard for implementing the IDL interface?
  • What crate/lib/types should be used?
7 Likes

I have a few thoughts about this. The first one is: what will typed bytecode (BTF) give us, and does it cover any things that IDLs cover?

1 Like

I really like the shank + anchor IDL format. Specifically, I like the fact that it makes the discriminant (i.e. instruction identifier) explicit. One of the nice things (some may disagree) about Solana is how much control it gives to the developer to come up with the semantics of how a program instruction should be dispatched. Anchor makes a lot of opinionated decisions on behalf of the devs, which ends up improving devex + security and reducing boilerplate, but I think those decisions should be explicit in the IDL rather than inferred from the client.

IMO the IDL should explicitly have 100% of the information needed to generate an instruction from a client as opposed forcing the client to read the instruction name and compute some bespoke hash before generating the instruction.

Note that these IDLs can never capture 100% of all execution paths because there is a bunch of custom logic that can be injected in a program to dictate how accounts should be deserialized. However, the shank + anchor IDL seems usable for like 99% of programs, so I think it makes sense as a standard.

4 Likes

For reference (coming from twitter thread) something that I had previously helped work on related to this topic:

2 Likes

This is where the gray area comes into play, though.

I agree Shank/Anchor IDLs cover a wide range of use cases and would be a great start for an interface, but what if you start to see an increase in alternate ways of deserializing accounts or instructions?
How do we begin to lobby indexers and tools to support these added configs in the IDL?
We may need to - at that time - introduce an interface/standard on top of an interface :exploding_head:

Something like deserializing data is integral to a lot of the tools that would need to know about these added configs, so that’s a use case where you might weigh “should ser/deser configs go into every IDL then? Or just ones that vary from the norm? What is the norm anyhow?”

A different custom config to an IDL - such as details on how your PDA’s addresses relate to each other (ie. seeds expanded) might be something that matters less to an indexer/explorer/UI of sorts. In this case, it might make more sense for the developer(s) and their amassed following to lobby these tooling providers to support their custom configs for some value-add reason, but ultimately it’s no sweat to the indexer tools if they don’t.

1 Like

I guess I’ll be the one to ask what you’re talking about, since I don’t know lol. Care to elaborate?

1 Like
1 Like

Do we know the timeline for ABIv2?
I don’t really see IDL interfacing/standardizing as a huge lift compared to other standards, so it might not hurt to sort out despite this impending change. Timeline-dependent, of course.

1 Like

While working on soda, have given significant consideration to the standardization of IDLs within the Solana ecosystem. I often find myself contemplating an expanded version of an IDL, including supplementary information that enables the incorporation of specific features. However, it is crucial to address a fundamental question: What is the primary purpose of an IDL, and what should it encompass? With this in mind, I believe we should carefully consider the following points:

Should the IDL strictly adhere to its namesake and serve solely as an interface description, or should it serve additional purposes?
Is it necessary to make seed information a mandatory component of the IDL?
Should the IDL incorporate information about the inner workings of the smart contract in some manner?
While I don’t possess definitive answers to these questions, I firmly believe it is imperative to establish clear boundaries regarding the purpose of an IDL. Instead of blindly adding various elements to the IDL, we should explore alternative sources and leverage additional data structures when necessary. If every developer starts incorporating the keys they deem necessary, we run the risk of ending up with divergent and non-standardized implementations that result in overlapping keys.