Tokenize texts

POST 
/v1/tokenize

By given a text input, create a tokenized output with token IDs.

Request

application/json

Body

prompt stringrequired

Input text prompt to tokenize.

Responses

200

Successfully tokenized the text.

application/json

Schema

Schema

tokens integer[]

A list of token IDs.

Example

Tokenized output.

{
  "tokens": [
    1724,
    338,
    2431,
    29875,
    17907,
    29973
  ]
}

Loading...