Using Execution API

Revised May 17, 2024 16:53

Introduction

With the Coginiti Execution API, users can effortlessly fetch data sets defined in the analytics catalog, bringing ease of use along with increased governance. It is designed to be compatible with popular data visualization and analysis tools such as PowerBI, Excel, and Jupyter Notebooks. It also could be used with popular orchestration tools such as Dagster or Kestra to trigger CoginitiScript publications or run data quality tests, ensuring both data consistency and reliability.

Base URL

All API URLs referenced in this documentation have the following base URL unless otherwise specified:

{hostname}/api/v1

Authentication

The following API requires authentication with a personal access token created in the Coginiti UI. Please follow the instructions here on how to create a personal access token.

Executing catalog assets to get a single result set

The given endpoint is used to execute a given script from the catalog and get a single result set. If the script produces several result sets, for example, when there are multiple SELECT statements present, only the first one will be executed and returned to the client.

Endpoint: {hostname}/api/v1/exec/script

Type of request: POST

Result format: JSON

Request example using curl application:

curl -X POST '{hostname}/api/v1/exec/script' \
  --header 'Authorization: Bearer _YOUR_ACCESS_TOKEN_HERE_' \
  --header 'Content-Type: application/json' \
  --header 'Accept: application/json' \
  --data-raw '{
  "path": "@Personal/Reports/performance",
  "params": {
    "start_date": "2012-01-01",
    "end_date": "2012-12-31"
  },
  "connection": {
    "name": "My Connection"
  }
}'

Body payload:

name	description
path	Absolute path to the catalog asset within the catalog. "@Personal", "@Shared", "@Legacy" are used to specify catalog namespace.
params	Object with parameter names and their values that need to be passed into the script to be substituted instead of $params.
connection.name	Name of the connection

Returns an object which represents result of the execution or an error otherwise.

name	description
stats	Statistics object.
stats.started_at	Timestamp when execution started.
stats.finished_at	Timestamp when execution is finished.
stats.duration_ms	Duration of execution in milliseconds.
result	Optional field. Contains result dataset if execution returns any data.

The dataset object has the following structure:

name	description
columns	A list of the names and types of the columns returned by the query.
data	A list of the rows returned by the query request. Each row is itself a list that holds values of the columns in the row, in the order specified by the columns attribute.

The additional metadata associated with the given execution is passed as HTTP headers.

name	description
X-Coginiti-Execution-Started-At	Timestamp when execution started

Example of the response (including HTTP headers):

HTTP/2 200 
Date: Wed, 01 Nov 2023 10:03:17 GMT
Content-Type: application/json
X-Coginiti-Execution-Started-At: 2023-11-01T10:03:17.354799859Z
Vary: Accept-Encoding
Server: envoy

{
    "result": {
        "columns": [
            {
                "name": "col_1",
                "type": "INTEGER"
            },
            {
                "name": "col_2",
                "type": "STRING"
            }
        ],
        "data": [
            [
                1,
                "a"
            ],
            [
                2,
                "b"
            ]
        ]
    },
    "stats": {
        "started_at": "2023-08-10T15:06:57.581939751Z",
        "finished_at": "2023-08-10T15:07:57.581939751Z",
        "duration_ms":  1000
    }
}

Executing CoginitiScript block (available from version 24.4)

The given endpoint is used to execute a given CoginitiScript block from the catalog asset and get a single result set.

Endpoint: {hostname}/api/v1/exec/block

Type of request: POST

Result format: JSON

Request example using curl application:

curl -X POST '{hostname}/api/v1/exec/block' \
  --header 'Authorization: Bearer _YOUR_ACCESS_TOKEN_HERE_' \
  --header 'Content-Type: application/json' \
  --header 'Accept: application/json' \
  --data-raw '{
  "package": "@Personal/My Projects/demo/customers",
  "block_name": "CustomersWithEmailDomain",
  "args": {
    "domain": "coginiti.co"
  },
  "params": {},
  "connection": {
    "name": "My Connection"
  }
}'

Body payload:

name	description
package	Absolute path to the CoginitiScript package within the catalog. "@Personal", "@Shared", "@Legacy" are used to specify catalog namespace.
block_name	Name of the block to execute within the specified package.
args	Object with argument names and values to pass into the specified block.
params	Object with parameter names and their values that need to be passed into the script to be substituted instead of $params.
connection.name	Name of the connection

Returns an object that represents the result of the execution or an error otherwise.

name	description
stats	Statistics object.
stats.started_at	Timestamp when execution started.
stats.finished_at	Timestamp when execution is finished.
stats.duration_ms	Duration of execution in milliseconds.
result	Optional field. Contains result dataset if execution returns any data.

The dataset object has the following structure:

name	description
columns	A list of the names and types of the columns returned by the query.
data	A list of the rows returned by the query request. Each row is itself a list that holds values of the columns in the row, in the order specified by the columns attribute.

The additional metadata associated with the given execution is passed as HTTP headers.

name	description
X-Coginiti-Execution-Started-At	Timestamp when execution started

Example of the response (including HTTP headers):

HTTP/2 200 
Date: Wed, 01 Nov 2023 10:03:17 GMT
Content-Type: application/json
X-Coginiti-Execution-Started-At: 2023-11-01T10:03:17.354799859Z
Vary: Accept-Encoding
Server: envoy

{
    "result": {
        "columns": [
            {
                "name": "col_1",
                "type": "INTEGER"
            },
            {
                "name": "col_2",
                "type": "STRING"
            }
        ],
        "data": [
            [
                1,
                "a"
            ],
            [
                2,
                "b"
            ]
        ]
    },
    "stats": {
        "started_at": "2023-08-10T15:06:57.581939751Z",
        "finished_at": "2023-08-10T15:07:57.581939751Z",
        "duration_ms":  1000
    }
}

Passing lists/maps as values to the block arguments

It is possible to pass not only literal values (numeric or string) into the block arguments but also composite types like lists and maps. To do this users have to use JSON list/object values.

Passing a list of values into "domains" block argument:

curl -X POST '{hostname}/api/v1/exec/block' \
  --header 'Authorization: Bearer _YOUR_ACCESS_TOKEN_HERE_' \
  --header 'Content-Type: application/json' \
  --header 'Accept: application/json' \
  --data-raw '{
  "package": "@Personal/My Projects/demo/customers",
  "block_name": "CustomersWithEmailDomain",
  "args": {
    "domains": ["coginiti.co", "gmail.com"]
  },
  "params": {},
  "connection": {
    "name": "My Connection"
  }
}'

Passing a map into "payment_fees" block argument:

curl -X POST '{hostname}/api/v1/exec/block' \
  --header 'Authorization: Bearer _YOUR_ACCESS_TOKEN_HERE_' \
  --header 'Content-Type: application/json' \
  --header 'Accept: application/json' \
  --data-raw '{
  "package": "@Personal/My Projects/demo/customers",
  "block_name": "CustomersWithEmailDomain",
  "args": {
    "payment_fees": {
      "credit_card": 0.25,
      "paypal": 0.3
    },
  },
  "params": {},
  "connection": {
    "name": "My Connection"
  }
}'

Specifying response format for single result

The Accept request HTTP header can be used by clients to specify in what format they want the result to be returned by the API. Supported options are:

application/json (used as a default option when no Accept header is specified)
text/csv
application/vnd.apache.arrow.stream