Table recognition

You can recognize text in a table using the OCR API with the table recognition model. With this model, you can recognize tables in Russian, English, and Turkish.

Getting started

To use the examples, install cURL.

Get your account data for authentication:

  1. Get an IAM token for your Yandex account, federated account, or local account.
  2. Get the ID of the folder for which your account has the ai.vision.user role or higher.
  3. When accessing Vision OCR via the API, provide the received parameters in each request:
  • Specify the IAM token in the Authorization header.
  • Specify the folder ID in the x-folder-id header.
Authorization: Bearer <IAM_token>
        x-folder-id: <folder_ID>

Vision OCR supports two authentication methods based on service accounts:

  • With an IAM token:

    1. Get an IAM token.

    2. Provide the IAM token in the Authorization header in the following format:

      Authorization: Bearer <IAM_token>

  • With API keys.

    Use API keys if requesting an IAM token automatically is not an option.

    1. Get an API key.

    2. Provide the API key in the Authorization header in the following format:

      Authorization: Api-Key <API_key>

Do not specify the folder ID in your requests, as Translate uses the folder where the service account was created.

Recognizing a table

Image text recognition is implemented in the recognize OCR API method.

  1. Prepare an image file that meets the requirements:

    • The supported file formats are JPEG, PNG, and PDF. Specify the MIME type of the file in the mime_type property. The default value is image.
    • The maximum file size is 10 MB.
    • The image size should not exceed 20 MP (height × width).

  2. Encode the image file as Base64:

    base64 -i input.jpg > output.txt
        
    C:> Base64.exe -e input.jpg > output.txt
        
    [Convert]::ToBase64String([IO.File]::ReadAllBytes("./input.jpg")) > output.txt
        
    # Import a library for encoding files in Base64.
        import base64
        
        # Create a function to encode a file and return the results.
        def encode_file(file_path):
          with open(file_path, "rb") as fid:
            file_content = fid.read()
          return base64.b64encode(file_content).decode("utf-8")
        
    // Read the file contents to memory.
        var fs = require('fs');
        var file = fs.readFileSync('/path/to/file');
        
        // Get the file contents in Base64 format.
        var encoded = Buffer.from(file).toString('base64');
        
    // Import a library for encoding files in Base64.
        import org.apache.commons.codec.binary.Base64;
        
        // Get the file contents in Base64 format.
        byte[] fileData = Base64.encodeBase64(yourFile.getBytes());
        
    import (
          "bufio"
          "encoding/base64"
          "io/ioutil"
          "os"
        )
        
        // Open the file.
        f, _ := os.Open("/path/to/file")
        
        // Read the file contents.
        reader := bufio.NewReader(f)
        content, _ := ioutil.ReadAll(reader)
        
        // Get the file contents in Base64 format.
        base64.StdEncoding.EncodeToString(content)

  3. Create a file with the request body, e.g., body.json.

    body.json:

    {
          "mimeType": "JPEG",
          "languageCodes": ["ru","en"],
          "model": "table",
          "content": "<base64-encoded_image>"
        }

    In the content property, specify the image file contents encoded as Base64.

  4. Send a request using the recognize method and save the response to a file, e.g., output.json:

    export IAM_TOKEN=<IAM_token>
        curl \
          --request POST \
          --header "Content-Type: application/json" \
          --header "Authorization: Bearer ${IAM_TOKEN}" \
          --header "x-folder-id: <folder_ID>" \
          --header "x-data-logging-enabled: true" \
          --data '{
            "mimeType": "JPEG",
            "languageCodes": ["ru","en"],
            "model": "handwritten",
            "content": "<base64_encoded_image>"
          }' \
          https://ocr.api.cloud.yandex.net/ocr/v1/recognizeText \
          --output output.json

    Where:

    • <IAM_token>: Previously obtained IAM token.
    • <folder_ID>: Previously obtained folder ID.
    data = {"mimeType": <mime_type>,
                "languageCodes": ["ru","en"],
                "content": content}
        
        url = "https://ocr.api.cloud.yandex.net/ocr/v1/recognizeText"
        
        headers= {"Content-Type": "application/json",
                  "Authorization": "Bearer {:s}".format(<IAM_token>),
                  "x-folder-id": "<folder_ID>",
                  "x-data-logging-enabled": "true"}
          
        w = requests.post(url=url, headers=headers, data=json.dumps(data))

    The result will consist of recognized blocks of text, lines, and words with their position on the image.

    Result:
    {
          "result": {
            "textAnnotation": {
              "width": "724",
              "height": "131",
              "blocks": [
                {
                  "boundingBox": {
                    "vertices": [
                      {
                        "x": "68",
                        "y": "42"
                      },
                      {
                        "x": "68",
                        "y": "60"
                      },
                      {
                        "x": "194",
                        "y": "60"
                      },
                      {
                        "x": "194",
                        "y": "42"
                      }
                    ]
                  },
                  "lines": [
                    {
                      "boundingBox": {
                        "vertices": [
                          {
                            "x": "68",
                            "y": "42"
                          },
                          {
                            "x": "68",
                            "y": "60"
                          },
                          {
                            "x": "194",
                            "y": "60"
                          },
                          {
                            "x": "194",
                            "y": "42"
                          }
                        ]
                      },
                      "text": "Фамилия",
                      "words": [
                        {
                          "boundingBox": {
                            "vertices": [
                              {
                                "x": "68",
                                "y": "40"
                              },
                              {
                                "x": "68",
                                "y": "63"
                              },
                              {
                                "x": "194",
                                "y": "63"
                              },
                              {
                                "x": "194",
                                "y": "40"
                              }
                            ]
                          },
                          "text": "Фамилия",
                          "entityIndex": "-1",
                          "textSegments": [
                            {
                              "startIndex": "0",
                              "length": "7"
                            }
                          ]
                        }
                      ],
                      "textSegments": [
                        {
                          "startIndex": "0",
                          "length": "7"
                        }
                      ]
                    }
                  ],
                  "languages": [
                    {
                      "languageCode": "ru"
                    }
                  ],
                  "textSegments": [
                    {
                      "startIndex": "0",
                      "length": "7"
                    }
                  ]
                },
                {
                  "boundingBox": {
                    "vertices": [
                      {
                        "x": "307",
                        "y": "40"
                      },
                      {
                        "x": "307",
                        "y": "61"
                      },
                      {
                        "x": "372",
                        "y": "61"
                      },
                      {
                        "x": "372",
                        "y": "40"
                      }
                    ]
                  },
                  "lines": [
                    {
                      "boundingBox": {
                        "vertices": [
                          {
                            "x": "307",
                            "y": "40"
                          },
                          {
                            "x": "307",
                            "y": "61"
                          },
                          {
                            "x": "372",
                            "y": "61"
                          },
                          {
                            "x": "372",
                            "y": "40"
                          }
                        ]
                      },
                      "text": "Имя",
                      "words": [
                        {
                          "boundingBox": {
                            "vertices": [
                              {
                                "x": "307",
                                "y": "39"
                              },
                              {
                                "x": "307",
                                "y": "63"
                              },
                              {
                                "x": "372",
                                "y": "63"
                              },
                              {
                                "x": "372",
                                "y": "39"
                              }
                            ]
                          },
                          "text": "Имя",
                          "entityIndex": "-1",
                          "textSegments": [
                            {
                              "startIndex": "8",
                              "length": "3"
                            }
                          ]
                        }
                      ],
                      "textSegments": [
                        {
                          "startIndex": "8",
                          "length": "3"
                        }
                      ]
                    }
                  ],
                  "languages": [
                    {
                      "languageCode": "ru"
                    }
                  ],
                  "textSegments": [
                    {
                      "startIndex": "8",
                      "length": "3"
                    }
                  ]
                },
                {
                  "boundingBox": {
                    "vertices": [
                      {
                        "x": "506",
                        "y": "43"
                      },
                      {
                        "x": "506",
                        "y": "59"
                      },
                      {
                        "x": "632",
                        "y": "59"
                      },
                      {
                        "x": "632",
                        "y": "43"
                      }
                    ]
                  },
                  "lines": [
                    {
                      "boundingBox": {
                        "vertices": [
                          {
                            "x": "506",
                            "y": "43"
                          },
                          {
                            "x": "506",
                            "y": "59"
                          },
                          {
                            "x": "632",
                            "y": "59"
                          },
                          {
                            "x": "632",
                            "y": "43"
                          }
                        ]
                      },
                      "text": "Отчество",
                      "words": [
                        {
                          "boundingBox": {
                            "vertices": [
                              {
                                "x": "506",
                                "y": "40"
                              },
                              {
                                "x": "506",
                                "y": "63"
                              },
                              {
                                "x": "632",
                                "y": "63"
                              },
                              {
                                "x": "632",
                                "y": "40"
                              }
                            ]
                          },
                          "text": "Отчество",
                          "entityIndex": "-1",
                          "textSegments": [
                            {
                              "startIndex": "12",
                              "length": "8"
                            }
                          ]
                        }
                      ],
                      "textSegments": [
                        {
                          "startIndex": "12",
                          "length": "8"
                        }
                      ]
                    }
                  ],
                  "languages": [
                    {
                      "languageCode": "ru"
                    }
                  ],
                  "textSegments": [
                    {
                      "startIndex": "12",
                      "length": "8"
                    }
                  ]
                },
                {
                  "boundingBox": {
                    "vertices": [
                      {
                        "x": "80",
                        "y": "81"
                      },
                      {
                        "x": "80",
                        "y": "98"
                      },
                      {
                        "x": "179",
                        "y": "98"
                      },
                      {
                        "x": "179",
                        "y": "81"
                      }
                    ]
                  },
                  "lines": [
                    {
                      "boundingBox": {
                        "vertices": [
                          {
                            "x": "80",
                            "y": "81"
                          },
                          {
                            "x": "80",
                            "y": "98"
                          },
                          {
                            "x": "179",
                            "y": "98"
                          },
                          {
                            "x": "179",
                            "y": "81"
                          }
                        ]
                      },
                      "text": "Иванов",
                      "words": [
                        {
                          "boundingBox": {
                            "vertices": [
                              {
                                "x": "80",
                                "y": "79"
                              },
                              {
                                "x": "80",
                                "y": "101"
                              },
                              {
                                "x": "179",
                                "y": "101"
                              },
                              {
                                "x": "179",
                                "y": "79"
                              }
                            ]
                          },
                          "text": "Иванов",
                          "entityIndex": "-1",
                          "textSegments": [
                            {
                              "startIndex": "21",
                              "length": "6"
                            }
                          ]
                        }
                      ],
                      "textSegments": [
                        {
                          "startIndex": "21",
                          "length": "6"
                        }
                      ]
                    }
                  ],
                  "languages": [
                    {
                      "languageCode": "ru"
                    }
                  ],
                  "textSegments": [
                    {
                      "startIndex": "21",
                      "length": "6"
                    }
                  ]
                },
                {
                  "boundingBox": {
                    "vertices": [
                      {
                        "x": "302",
                        "y": "80"
                      },
                      {
                        "x": "302",
                        "y": "98"
                      },
                      {
                        "x": "370",
                        "y": "98"
                      },
                      {
                        "x": "370",
                        "y": "80"
                      }
                    ]
                  },
                  "lines": [
                    {
                      "boundingBox": {
                        "vertices": [
                          {
                            "x": "302",
                            "y": "80"
                          },
                          {
                            "x": "302",
                            "y": "98"
                          },
                          {
                            "x": "370",
                            "y": "98"
                          },
                          {
                            "x": "370",
                            "y": "80"
                          }
                        ]
                      },
                      "text": "Иван",
                      "words": [
                        {
                          "boundingBox": {
                            "vertices": [
                              {
                                "x": "302",
                                "y": "78"
                              },
                              {
                                "x": "302",
                                "y": "101"
                              },
                              {
                                "x": "370",
                                "y": "101"
                              },
                              {
                                "x": "370",
                                "y": "78"
                              }
                            ]
                          },
                          "text": "Иван",
                          "entityIndex": "-1",
                          "textSegments": [
                            {
                              "startIndex": "28",
                              "length": "4"
                            }
                          ]
                        }
                      ],
                      "textSegments": [
                        {
                          "startIndex": "28",
                          "length": "4"
                        }
                      ]
                    }
                  ],
                  "languages": [
                    {
                      "languageCode": "ru"
                    }
                  ],
                  "textSegments": [
                    {
                      "startIndex": "28",
                      "length": "4"
                    }
                  ]
                },
                {
                  "boundingBox": {
                    "vertices": [
                      {
                        "x": "504",
                        "y": "82"
                      },
                      {
                        "x": "504",
                        "y": "98"
                      },
                      {
                        "x": "630",
                        "y": "98"
                      },
                      {
                        "x": "630",
                        "y": "82"
                      }
                    ]
                  },
                  "lines": [
                    {
                      "boundingBox": {
                        "vertices": [
                          {
                            "x": "504",
                            "y": "82"
                          },
                          {
                            "x": "504",
                            "y": "98"
                          },
                          {
                            "x": "630",
                            "y": "98"
                          },
                          {
                            "x": "630",
                            "y": "82"
                          }
                        ]
                      },
                      "text": "Иванович",
                      "words": [
                        {
                          "boundingBox": {
                            "vertices": [
                              {
                                "x": "504",
                                "y": "80"
                              },
                              {
                                "x": "504",
                                "y": "101"
                              },
                              {
                                "x": "630",
                                "y": "101"
                              },
                              {
                                "x": "630",
                                "y": "80"
                              }
                            ]
                          },
                          "text": "Иванович",
                          "entityIndex": "-1",
                          "textSegments": [
                            {
                              "startIndex": "33",
                              "length": "8"
                            }
                          ]
                        }
                      ],
                      "textSegments": [
                        {
                          "startIndex": "33",
                          "length": "8"
                        }
                      ]
                    }
                  ],
                  "languages": [
                    {
                      "languageCode": "ru"
                    }
                  ],
                  "textSegments": [
                    {
                      "startIndex": "33",
                      "length": "8"
                    }
                  ]
                }
              ],
              "entities": [],
              "tables": [
                {
                  "boundingBox": {
                    "vertices": [
                      {
                        "x": "34",
                        "y": "30"
                      },
                      {
                        "x": "687",
                        "y": "41"
                      },
                      {
                        "x": "686",
                        "y": "119"
                      },
                      {
                        "x": "33",
                        "y": "108"
                      }
                    ]
                  },
                  "rowCount": "2",
                  "columnCount": "3",
                  "cells": [
                    {
                      "boundingBox": {
                        "vertices": [
                          {
                            "x": "35",
                            "y": "30"
                          },
                          {
                            "x": "242",
                            "y": "33"
                          },
                          {
                            "x": "241",
                            "y": "64"
                          },
                          {
                            "x": "34",
                            "y": "61"
                          }
                        ]
                      },
                      "rowIndex": "0",
                      "columnIndex": "0",
                      "columnSpan": "1",
                      "rowSpan": "1",
                      "text": "Фамилия",
                      "textSegments": [
                        {
                          "startIndex": "0",
                          "length": "7"
                        }
                      ]
                    },
                    {
                      "boundingBox": {
                        "vertices": [
                          {
                            "x": "242",
                            "y": "33"
                          },
                          {
                            "x": "444",
                            "y": "37"
                          },
                          {
                            "x": "444",
                            "y": "67"
                          },
                          {
                            "x": "241",
                            "y": "64"
                          }
                        ]
                      },
                      "rowIndex": "0",
                      "columnIndex": "1",
                      "columnSpan": "1",
                      "rowSpan": "1",
                      "text": "Имя",
                      "textSegments": [
                        {
                          "startIndex": "8",
                          "length": "3"
                        }
                      ]
                    },
                    {
                      "boundingBox": {
                        "vertices": [
                          {
                            "x": "444",
                            "y": "37"
                          },
                          {
                            "x": "686",
                            "y": "40"
                          },
                          {
                            "x": "685",
                            "y": "71"
                          },
                          {
                            "x": "444",
                            "y": "67"
                          }
                        ]
                      },
                      "rowIndex": "0",
                      "columnIndex": "2",
                      "columnSpan": "1",
                      "rowSpan": "1",
                      "text": "Отчество",
                      "textSegments": [
                        {
                          "startIndex": "12",
                          "length": "8"
                        }
                      ]
                    },
                    {
                      "boundingBox": {
                        "vertices": [
                          {
                            "x": "34",
                            "y": "61"
                          },
                          {
                            "x": "241",
                            "y": "64"
                          },
                          {
                            "x": "241",
                            "y": "108"
                          },
                          {
                            "x": "34",
                            "y": "105"
                          }
                        ]
                      },
                      "rowIndex": "1",
                      "columnIndex": "0",
                      "columnSpan": "1",
                      "rowSpan": "1",
                      "text": "Иванов",
                      "textSegments": [
                        {
                          "startIndex": "21",
                          "length": "6"
                        }
                      ]
                    },
                    {
                      "boundingBox": {
                        "vertices": [
                          {
                            "x": "241",
                            "y": "64"
                          },
                          {
                            "x": "444",
                            "y": "67"
                          },
                          {
                            "x": "443",
                            "y": "111"
                          },
                          {
                            "x": "241",
                            "y": "108"
                          }
                        ]
                      },
                      "rowIndex": "1",
                      "columnIndex": "1",
                      "columnSpan": "1",
                      "rowSpan": "1",
                      "text": "Иван",
                      "textSegments": [
                        {
                          "startIndex": "28",
                          "length": "4"
                        }
                      ]
                    },
                    {
                      "boundingBox": {
                        "vertices": [
                          {
                            "x": "444",
                            "y": "67"
                          },
                          {
                            "x": "685",
                            "y": "71"
                          },
                          {
                            "x": "684",
                            "y": "115"
                          },
                          {
                            "x": "443",
                            "y": "111"
                          }
                        ]
                      },
                      "rowIndex": "1",
                      "columnIndex": "2",
                      "columnSpan": "1",
                      "rowSpan": "1",
                      "text": "Иванович",
                      "textSegments": [
                        {
                          "startIndex": "33",
                          "length": "8"
                        }
                      ]
                    }
                  ]
                }
              ],
              "fullText": "Фамилия\nИмя\nОтчество\nИванов\nИван\nИванович\n"
            },
            "page": "0"
          }
        }

  5. To get all the recognized words in an image, find all the values with the text property.

Note

If the coordinates you got do not match the position of displayed elements, set up support for exif metadata in your image viewing tool or remove the Orientation attribute from the exif image section when running a transfer to the service.

Previous
Handwriting recognition
Next
Base64 encoding