Sending an asynchronous request

To use the examples of requests using SDK:

1. Create a service account and assign it the ai.languageModels.user role.

2. Get and save the service account's API key with yc.ai.foundationModels.execute for its scope.

   The following examples use API key authentication. Yandex Cloud ML SDK also supports IAM token and OAuth token authentication. For more information, see Authentication in Yandex Cloud ML SDK.

   Note

   If you are using Windows, we recommend installing the WSL shell first and using it to proceed.

3. Install Python 3.10 or higher.

4. Optionally, install Python venv to create isolated virtual environments in Python.

5. Optionally, create a new Python virtual environment and activate it:

   python3 -m venv new-env
           source new-env/bin/activate
           

6. Use the pip package manager to install the ML SDK library:

   pip install yandex-ai-studio-sdk
           

When using Yandex Cloud ML SDK, you can configure your code to wait for the operation to complete and return the response. To do this, use either the sleep function of the time module or the wait method. The example utilizes both of these methods one by one.

1. Create a file named generate-deferred.py and paste the following code into it:

   #!/usr/bin/env python3
           
           from __future__ import annotations
           import time
           from yandex_ai_studio_sdk import AIStudio
           
           messages_1 = [
               {
                   "role": "system",
                   "text": "Find errors in the text and correct them",
               },
               {
                   "role": "user",
                   "text": """Laminate flooring is sutiable for instalation in the kitchen or in a child's
           room. It withsatnds moisturre and mechanical dammage thanks to 
           a 0.2 mm thick proctive layer of melamine films and 
           a wax-treated interlocking system.""",
               },
           ]
           
           messages_2 = [
               {"role": "system", "text": "Find errors in the text and correct them"},
               {"role": "user", "text": "Erors wyll not corrct themselfs."},
           ]
           
           
           def main():
           
               sdk = AIStudio(
                   folder_id="<folder_ID>",
                   auth="<API_key>",
               )
           
               model = sdk.models.completions("yandexgpt")
           
               # Variant 1: wait for the operation to complete using 5-second sleep periods
           
               print("Variant 1:")
           
               operation = model.configure(temperature=0.5).run_deferred(messages_1)
           
               status = operation.get_status()
               while status.is_running:
                   time.sleep(5)
                   status = operation.get_status()
           
               result = operation.get_result()
               print(result)
           
               # Variant 2: wait for the operation to complete using the wait method
           
               print("Variant 2:")
           
               operation = model.run_deferred(messages_2)
           
               result = operation.wait()
               print(result)
           
           
           if __name__ == "__main__":
               main()
           

   Where:

   Note

   As input data for a request, Yandex Cloud ML SDK can accept a string, a dictionary, an object of the TextMessage class, or an array containing any combination of these data types. For more information, see Yandex Cloud ML SDK usage.

   • messages_1 and messages_2: Arrays of messages providing the context for the model, each used for a different method of getting an asynchronous request result:

     • role: Message sender's role:

       • user: To send user messages to the model.
       • system: To set the request context and define the model's behavior.
       • assistant: For responses generated by the model. In chat mode, the model's responses tagged with the assistant role are included in the message to save the conversation context. Do not send user messages with this role.

   • text: Message text.

   • YANDEX_FOLDER_ID: Service account folder ID.

   • YANDEX_API_KEY: API key for access to AI Studio.

   • YANDEX_TUNING_SUFFIX: Model tuning suffix.

     The following examples use API key authentication. Yandex Cloud ML SDK also supports IAM token and OAuth token authentication. For more information, see Authentication in Yandex Cloud ML SDK.

   For more information about accessing a specific model version, see {#T}.

2. Run the file you created:

   python3 generate-deferred.py
           

   Result:

   Variant 1:
           GPTModelResult(alternatives=(Alternative(role='assistant', text='Laminate flooring is suitable for installation in the kitchen or in a child's room. It withstands moisture and mechanical damage thanks to a 0.2 mm thick protective layer of melamine films and a wax-treated interlocking system.', status=<AlternativeStatus.FINAL: 3>),), usage=Usage(input_text_tokens=74, completion_tokens=46, total_tokens=120), model_version='23.10.2024')
           Variant 2:
           GPTModelResult(alternatives=(Alternative(role='assistant', text='Errors will not correct themselves.\n\nErors → errors.', status=<AlternativeStatus.FINAL: 3>),), usage=Usage(input_text_tokens=32, completion_tokens=16, total_tokens=48), model_version='23.10.2024')
           

   The code waits for the result of the first method and then of the second one.