sRFC 33: Media Types of Blink

sRFC
1

Summary

Currently, the media types, support for the icon field are svg , png, webp, which limits the design space for the developer to show content.

The goal is to introduce default media types supported by most of the latest browsers, which will widen the type of content that blinks can show starting with video/audio and more types of images.

Media Types

1. Image Types
  • JPEG (image/jpeg)
  • GIF (image/gif)
  • BMP (image/bmp)
2. Audio Types
  • MP3 (audio/mpeg)
  • WAV (audio/wav)
  • OGG (audio/ogg)
  • AAC (audio/aac)
  • WebM Audio (audio/webm)
3. Video Types
  • MP4 (video/mp4)
  • WebM (video/webm)
  • OGG Video (video/ogg)
  • MPEG (video/mpeg)

Implementation

Currently the ActionGetResponse looks like below

/**
 * Response body payload returned from the initial Action GET Request
 *
 * note: `type` is optional for backwards compatibility
 */
export interface ActionGetResponse extends Omit<Action, "type"> {
  type?: "action";
}

/**
 * A single Solana Action
 */
export interface Action<T extends ActionType = "action"> {
  /** type of Action to present to the user */
  type: T;
  /** image url that represents the source of the action request */
  icon: string;
  /** describes the source of the action request */
  title: string;
  /** brief summary of the action to be performed */
  description: string;
  /** button text rendered to the user */
  label: string;
  /** UI state for the button being rendered to the user */
  disabled?: boolean;
  links?: {
    /** list of related Actions a user could perform */
    actions: LinkedAction[];
  };
  /** non-fatal error message to be displayed to the user */
  error?: ActionError;
}

If we modify the Action to include 2 additional fields, media and mediaType

/** 
* Media Types for media in the action
*/
export type MediaType = "video/mp4" | "video/webm" | "video/ogg" | "video/mpeg" | "image/jpeg" | "image/png" | "image/gif" | "image/svg+xml" | "image/webp" | "image/bmp" | "image/x-icon" | "audio/mpeg" | "audio/wav" | "audio/ogg" | "audio/aac" | "audio/webm"

/**
 * A single Solana Action
 */
export interface Action<T extends ActionType = "action"> {
  /** type of Action to present to the user */
  type: T;
  /** image url that represents the source of the action request, and
 will be used as a fallback image incase
 media URL is un-reachable/un-renderable */
  icon: string;
  /** media URL that represents the content to be shown upon request**/
  media : string;
  /** represents the type of media URL provided */
  mediaType : MediaType;
  /** describes the source of the action request */
  title: string;
  /** brief summary of the action to be performed */
  description: string;
  /** button text rendered to the user */
  label: string;
  /** UI state for the button being rendered to the user */
  disabled?: boolean;
  links?: {
    /** list of related Actions a user could perform */
    actions: LinkedAction[];
  };
  /** non-fatal error message to be displayed to the user */
  error?: ActionError;
}

Then this would allow the blink developer to include more media types, icon field would still be present to maintain backward compatibility + used as a fallback if the media URL is un-reachable / un-renderable.

1 Like
2

Very much agree the current media types can be limiting.

I do think perhaps not restricting to just one file though could be helpful and more forward compatible as spec evolves. Perhaps formatting the media field more like the optional links field would allow for this while also keeping the spec compact:

media?: {
    /** list of media items to display */
    items: MediaItem[];
  };

where media items could be:

interface MediaItem {
  url: string;
  type: MediaType;
}

Of course this brings up the question of how to render multiple media files. An initial thought is providing an optional display field could define how to show the media files with the default being simply displaying the first item.

media?: {
    items: MediaItem[];
    display?: "default" | "carousel" | "gallery" | "list";
  };
3

What use-case do you think for array of Media, that would make it a gallery like layout, which imo would not be required by majority of use case

4

This comes primarily from e-commerce use case where want to have multiple images of a product without forcing a user to create complex product images or having to custom generate one image from a set of images. I also think social, gaming, and RWA blinks would find this useful too.

3 Likes