QuickStart: Friendli Serverless Endpoints

1. Log In or Sign Up

• If you have an account, log in using your preferred SSO or email/password combination.
• If you're new to FriendliAI, create an account for free.

2. Access Friendli Serverless Endpoints

• On your dashboard, find the "Friendli Serverless Endpoints" section.
• Click the "Go to Playground" button to start generating text.

3. Select a Model

• Browse the available generative models.
• Choose the model that best aligns with your desired use case.
• First-time users receives a $5 free trial to explore Friendli Serverless Endpoints without any financial commitment.

4. Generate Responses

1. Enter Your Query:

• Type in your prompt or question.
• Alternatively, select from the provided example queries to try out different scenarios.

2. Adjust Settings:

• Refer to the Text Generation docs for more details on the settings applicable for the text generation models.

3. Generate Your Response:

• Click the "Generate" button to start the generation process.
• The model will process your query and produce the corresponding text output. That's it!

info

Generating Responses Through the Endpoint URL

If you wish to send your requests through the endpoint URL, you can find the endpoint URL by hitting the 'More Info` button on the top-right corner of the page. Refer to this guide for general instructions on the personal access tokens.

Bash (Text)

# Send inference request to a running Friendli Serverless Endpoint using a `curl` command.

$ curl -X POST https://inference.friendli.ai/v1/completions \
  -H "Authorization: Bearer $FRIENDLI_TOKEN" \
  -d '{"prompt": "Python is a popular", "min_tokens": 20, "max_tokens": 30,
       "top_k": 32, "top_p": 0.8, "n": 3, "no_repeat_ngram": 3,
       "ngram_repetition_penalty": 1.75}'

Bash (Image)

# Send inference request to a running Friendli Serverless Endpoint using a `curl` command.

$ curl -X POST https://inference.friendli.ai/v1/text-to-image \
  -H "Authorization: Bearer $FRIENDLI_TOKEN" \
  -F "prompt=$PROMPT" -F "num_inference_steps=25"

Python SDK

# pip install friendli-client
# Send inference request to a Friendli Serverless Endpoint using Python SDK.
import os

from friendli import Friendli

client = Friendli(token=os.getenv("FRIENDLI_TOKEN"))

chat_completion = client.chat.completions.create(
    model="meta-llama-3.1-70b-instruct",
    messages=[
        {
            "role": "user",
            "content": "Tell me how to make a delicious pancake"
        }
    ],
    stream=False,
)
print(chat_completion.choices[0].message.content)

Additional Tips:

Check out the Text Generation docs for more details.

Ready to unlock the creativity of generative AI? Get started with Friendli Serverless Endpoints today!