Overview

The Data Management API enables the following functionality by group:

Group	Description
Data Discovery	Discover data in the datalake.
Data Location	Retrieve access points for data in the datalake.
Data Access	Access data in the datalake.
Data Logistics	Adjust placement of data in the datalake.
Data Staging	Stage existing data somewhere else in the datalake
Metadata	Metadata operations.
Schemas	Schema operations.
Status	Operations describing the status of the API.

AuthN/Z

Authentication

User

To access this API as a user, the user needs to have first authenticated with the SRCNet and to have exchanged the token resulting from this initial authentication with one that allows access to this specific service. See the Authentication Mechanism and Token Exchange Mechanism sections of the Auth API for more specifics.

Service

For service-to-service interactions, it is possible to obtain a token via a client_credentials grant to the ska-src-data-management-api IAM client.

Authorisation

Hereafter, the caller (either a user or another service) is assumed to have a valid token allowing access to this API. Authenticated requests are then made by including this token in the header.

The token audience must also match the expected audience, also defined in the data-management-api permissions policy (default: “data-management-api”).

Restricting user access to routes using token scopes

The presented token must include a specific scope expected by the service to be permitted access to all API routes. This scope is defined in the data-management-api permissions policy (default: “data-management-api-service”).

This scope must also be added to the IAM permissions client otherwise the process of token instrospection will drop this scope.

Restricting user access to routes using IAM groups

Access to a specific route of this API depends on user IAM group membership and is determined by calls to the /authorise/route path of the Permissions API. Default groups are hierarchically nested with the pattern e.g. root_group/namespaces/{namespace}.

As an example, consider get/set metadata functionality. The metadata get/set routes are protected by the following permission policy mapping API routes to "roles":

{
    "/metadata/{namespace}/{name}": {
        "GET": "admin or namespace-viewer or namespace-editor or namespace-owner",
        "POST": "admin or namespace-editor or namespace-owner"
    },
}

where the namespace-viewer, namespace-editor and namespace-owner roles are only assigned for users who have the following IAM group membership:

{
    "namespace-viewer": [
        "{root_group}/namespaces/{namespace}/viewer"
    ],
    "namespace-editor": [
        "{root_group}/namespaces/{namespace}/editor"
    ],
    "namespace-owner": [
        "{root_group}/namespaces/{namespace}/owner"
    ],
}

So that the required group membership to access the get metadata endpoint for the e.g. testing namespace would look something like:

/services/data-management-api/
    testing/
        viewer||editor||owner

and likewise for the set metadata endpoint:

/services/data-management-api/
    testing/
        editor||owner

Roles are assigned when a request to a particular endpoint is made. This enables information from the request to be used to understand if a role can be assigned. For example, consider the namespace-viewer role:

    "namespace-viewer": [
        "{root_group}/namespaces/{namespace}/viewer"
    ],

which requires both root_group and namespace to be provided. The root_group is an application specific parameter, but the namespace parameter is substituted when the request is made. In the case of a GET request for metadata, the route /metadata/{namespace}/{name} provides the namespace as a path parameter, and this value is substituted into the role definition. The source of the substitution for the role definition depends on either the path parameters, query parameters or body of the request; which are used depends on where the parameters are expected to come from.