openapi: 3.0.3 info: title: Data Catalog Plugin API description: | A **Data Catalog Plugin** is a microservice that is responsible for provisioning and unprovisioning metadata into a Data Catalog during the deployment phase of a resource through Witboost. It takes in a system descriptor enriched with components' provisioning results and returns the result of the provisioned Data Catalog entities. The Data Catalog contract is similar to a technology adapter, with a few differences. It takes in the whole system descriptor already enriched with tech adapters' deploy info. version: '2.2.0' servers: - url: /datamesh.datacatalogplugin tags: - name: DataCatalogPlugin paths: /v2/validate: post: tags: - DataCatalogPlugin summary: Validate a Data Catalog Entity provision request (async) description: | Validate output ports metadata attached in the provisioning request. In particular, it validates the format and the existence of glossary terms and classification tags operationId: asyncValidate requestBody: description: Details of a provisioning request to be validated content: application/json: schema: $ref: '#/components/schemas/ProvisioningRequest' examples: systemDescriptor: value: descriptorKind: DATAPRODUCT_DESCRIPTOR descriptor: "dataProduct:\n id: urn:dmb:dp:finance:a-system:0\n components: \n - id: urn:dmb:dp:finance:a-system:0:component:0\n name: S3 storage\n description: A system that does something\n kind: storage\n version: 0.0.1\n - id: urn:dmb:dp:finance:a-system:0:component:1\n name: A system that does something\n description: A system that does something\n kind: outputport\n version: 0.0.1\nremoveData: false" removeData: false componentDescriptor: value: descriptorKind: COMPONENT_DESCRIPTOR descriptor: "dataProduct:\n id: urn:dmb:dp:finance:a-system:0\n components: \n - id: urn:dmb:dp:finance:a-system:0:component:0\n name: S3 storage\n description: A system that does something\n kind: storage\n version: 0.0.1\n - id: urn:dmb:dp:finance:a-system:0:component:1\n name: A system that does something\n description: A system that does something\n kind: outputport\n version: 0.0.1\ncomponentIdToProvision: urn:dmb:dp:finance:a-system:0:component:1" removeData: false required: true responses: 202: description: Validate Task has been accepted content: application/json: schema: type: string description: Validate Task token that can be used for polling the request status example: 2f4b3b3b-4b3b-4b3b-4b3b-4b3b4b3b4b3b 400: description: Invalid Request content: application/json: schema: $ref: "#/components/schemas/RequestValidationError" examples: missingComponentId: value: errors: - "Component ID is missing" userMessage: "Component ID is missing" input: "dataProduct:\n id: urn:dmb:dp:finance:a-system:0\n components: \n - id: urn:dmb:dp:finance:a-system:0:component:0\n name: S3 storage\n description: A system that does something\n kind: storage\n version: 0.0.1\n - id: urn:dmb:dp:finance:a-system:0:component:1\n name: A system that does something\n description: A system that does something\n kind: outputport\n version: 0.0.1\nremoveData: false" inputErrorField: "componentIdToProvision" moreInfo: problems: - "Component ID is missing" solutions: - "Please provide the component ID" 500: description: Internal Error content: application/json: schema: $ref: '#/components/schemas/SystemError' examples: internalError: value: error: "Internal Error" userMessage: "An internal error occurred" moreInfo: problems: - "An internal error occurred" solutions: - "Please try again later" /v2/validate/{token}/status: get: tags: - DataCatalogPlugin summary: Get status of an async validation task operationId: getValidationStatus parameters: - name: token in: path description: Token obtained after a call to the async validate endpoint required: true schema: type: string example: 2a4bb060-27c9-403d-ab5d-d3776cf56c3b responses: 200: description: Validation Task status content: application/json: schema: $ref: "#/components/schemas/ValidationStatus" examples: validationStatus: value: status: "COMPLETED" info: publicInfo: valid: true privateInfo: valid: true 400: description: Invalid Request content: application/json: schema: $ref: "#/components/schemas/RequestValidationError" examples: missingToken: value: errors: - "Token is missing" 500: description: Internal Error content: application/json: schema: $ref: "#/components/schemas/SystemError" examples: internalError: value: error: "Internal Error" userMessage: "An internal error occurred" moreInfo: problems: - "An internal error occurred" solutions: - "Please try again later" /v1/validate: post: tags: - DataCatalogPlugin summary: Validate a Data Catalog Entity provision request (sync) description: | Synchronously validate a provisioning request and return the validation result. You want to implement this endpoint to assess that the provided resource descriptor conforms to the expected structure and has all the necessary information to be deployed on the infrastructure. **It is highly recommended to implement the asynchronous validation endpoint instead of this one.** operationId: validate requestBody: description: Details of a provisioning request to be validated content: application/json: schema: $ref: "#/components/schemas/ProvisioningRequest" examples: systemDescriptor: value: descriptorKind: DATAPRODUCT_DESCRIPTOR descriptor: "dataProduct:\n id: urn:dmb:dp:finance:a-system:0\n components: \n - id: urn:dmb:dp:finance:a-system:0:component:0\n name: S3 storage\n description: A system that does something\n kind: storage\n version: 0.0.1\n - id: urn:dmb:dp:finance:a-system:0:component:1\n name: A system that does something\n description: A system that does something\n kind: outputport\n version: 0.0.1\nremoveData: false" removeData: false componentDescriptor: value: descriptorKind: COMPONENT_DESCRIPTOR descriptor: "dataProduct:\n id: urn:dmb:dp:finance:a-system:0\n components: \n - id: urn:dmb:dp:finance:a-system:0:component:0\n name: S3 storage\n description: A system that does something\n kind: storage\n version: 0.0.1\n - id: urn:dmb:dp:finance:a-system:0:component:1\n name: A system that does something\n description: A system that does something\n kind: outputport\n version: 0.0.1\ncomponentIdToProvision: urn:dmb:dp:finance:a-system:0:component:1" removeData: false required: true responses: 200: description: Validation Task result content: application/json: schema: $ref: "#/components/schemas/ValidationResult" examples: validDescriptor: value: valid: true 500: description: Internal Error content: application/json: schema: $ref: "#/components/schemas/SystemError" examples: internalError: value: error: "Internal Error" userMessage: "An internal error occurred" moreInfo: problems: - "An internal error occurred" solutions: - "Please try again later" /v1/provision: post: tags: - DataCatalogPlugin summary: Provision Data Catalog entity description: | Provisioning is the operation that deploys the metadata of a system or a single component into a third-party Data Catalog. This request can be handled in synchronous or asynchronous mode depending on the implementation. operationId: provision requestBody: description: A Data Product descriptor, enriched with components' provisioning results, wrapped as a string into a simple object content: application/json: schema: $ref: '#/components/schemas/ProvisioningRequest' examples: systemDescriptor: value: descriptorKind: DATAPRODUCT_DESCRIPTOR descriptor: "dataProduct:\n id: urn:dmb:dp:finance:a-system:0\n components: \n - id: urn:dmb:dp:finance:a-system:0:component:0\n name: S3 storage\n description: A system that does something\n kind: storage\n version: 0.0.1\n - id: urn:dmb:dp:finance:a-system:0:component:1\n name: A system that does something\n description: A system that does something\n kind: outputport\n version: 0.0.1\nremoveData: false" removeData: false componentDescriptor: value: descriptorKind: COMPONENT_DESCRIPTOR descriptor: "dataProduct:\n id: urn:dmb:dp:finance:a-system:0\n components: \n - id: urn:dmb:dp:finance:a-system:0:component:0\n name: S3 storage\n description: A system that does something\n kind: storage\n version: 0.0.1\n - id: urn:dmb:dp:finance:a-system:0:component:1\n name: A system that does something\n description: A system that does something\n kind: outputport\n version: 0.0.1\ncomponentIdToProvision: urn:dmb:dp:finance:a-system:0:component:1" removeData: false required: true responses: 200: description: Entity Provision result (sync) content: application/json: schema: $ref: '#/components/schemas/ProvisioningStatus' examples: provisionResult: value: status: "COMPLETED" info: publicInfo: entitiesCommitted: 10 privateInfo: entitiesCommitted: 10 glossaryId: dfasd13 logs: - timestamp: "2021-09-30T10:00:00Z" level: "INFO" message: "Entity provisioned successfully" 202: description: Entity Provision task has been accepted content: application/json: schema: type: string examples: provisionAccepted: value: "a3138dfc-07d4-4c05-8d81-e7d389bd19f8" 400: description: Invalid Request content: application/json: schema: $ref: '#/components/schemas/RequestValidationError' examples: missingComponentId: value: errors: - "Component ID is missing" userMessage: "Component ID is missing" input: "dataProduct:\n id: urn:dmb:dp:finance:a-system:0\n components: \n - id: urn:dmb:dp:finance:a-system:0:component:0\n name: S3 storage\n description: A system that does something\n kind: storage\n version: 0.0.1\n - id: urn:dmb:dp:finance:a-system:0:component:1\n name: A system that does something\n description: A system that does something\n kind: outputport\n version: 0.0.1\nremoveData: false" inputErrorField: "componentIdToProvision" moreInfo: problems: - "Component ID is missing" solutions: - "Please provide the component ID" 500: description: Internal Error content: application/json: schema: $ref: '#/components/schemas/SystemError' examples: internalError: value: error: "Internal Error" userMessage: "An internal error occurred" moreInfo: problems: - "An internal error occurred" solutions: - "Please try again later" /v1/unprovision: post: tags: - DataCatalogPlugin summary: Unprovision Data Catalog entity description: | Unprovisioning is the operation that removes the metadata of a system or a single component from a third-party Data Catalog. This request is synchronous and returns the result of the unprovisioning process. operationId: unprovision requestBody: description: A system descriptor and the provisioning results wrapped as simple object content: application/json: schema: $ref: '#/components/schemas/ProvisioningRequest' examples: systemDescriptor: value: descriptorKind: DATAPRODUCT_DESCRIPTOR descriptor: "dataProduct:\n id: urn:dmb:dp:finance:a-system:0\n components: \n - id: urn:dmb:dp:finance:a-system:0:component:0\n name: S3 storage\n description: A system that does something\n kind: storage\n version: 0.0.1\n - id: urn:dmb:dp:finance:a-system:0:component:1\n name: A system that does something\n description: A system that does something\n kind: outputport\n version: 0.0.1\nremoveData: false" removeData: false componentDescriptor: value: descriptorKind: COMPONENT_DESCRIPTOR descriptor: "dataProduct:\n id: urn:dmb:dp:finance:a-system:0\n components: \n - id: urn:dmb:dp:finance:a-system:0:component:0\n name: S3 storage\n description: A system that does something\n kind: storage\n version: 0.0.1\n - id: urn:dmb:dp:finance:a-system:0:component:1\n name: A system that does something\n description: A system that does something\n kind: outputport\n version: 0.0.1\ncomponentIdToProvision: urn:dmb:dp:finance:a-system:0:component:1" removeData: false required: true responses: 200: description: Unprovision Task result (sync) content: application/json: schema: $ref: '#/components/schemas/ProvisioningStatus' examples: unprovisionResult: value: status: "COMPLETED" info: publicInfo: entitiesRemoved: 10 privateInfo: entitiesRemoved: 10 glossaryId: dfasd13 logs: - timestamp: "2021-09-30T10:00:00Z" level: "INFO" message: "Entity unprovisioned successfully" 202: description: Unprovision Task has been accepted content: application/json: schema: type: string description: Unprovision Task token that can be used for polling the request status example: 8375e99f-0305-4005-8cce-4b5e1de435b3 400: description: Invalid Request content: application/json: schema: $ref: '#/components/schemas/RequestValidationError' examples: missingComponentId: value: errors: - "Component ID is missing" userMessage: "Component ID is missing" input: "dataProduct:\n id: urn:dmb:dp:finance:a-system:0\n components: \n - id: urn:dmb:dp:finance:a-system:0:component:0\n name: S3 storage\n description: A system that does something\n kind: storage\n version: 0.0.1\n - id: urn:dmb:dp:finance:a-system:0:component:1\n name: A system that does something\n description: A system that does something\n kind: outputport\n version: 0.0.1\nremoveData: false" inputErrorField: "componentIdToProvision" moreInfo: problems: - "Component ID is missing" solutions: - "Please provide the component ID" 500: description: Internal Error content: application/json: schema: $ref: '#/components/schemas/SystemError' examples: internalError: value: error: "Internal Error" userMessage: "An internal error occurred" moreInfo: problems: - "An internal error occurred" solutions: - "Please try again later" /v1/provision/{token}/status: get: tags: - DataCatalogPlugin summary: Get status of Data Catalog entity provision/unprovision task operationId: getStatus parameters: - name: token in: path description: Data Catalog Provision/Unprovision Task Token obtained after a call to the provision/unprovision endpoint required: true schema: type: string example: f6440015-1404-4e96-a250-7f680dbc32d4 responses: 200: description: Metadata Provisioning/Unprovisioning Task Status content: application/json: schema: $ref: "#/components/schemas/ProvisioningStatus" examples: provisionStatus: value: status: "COMPLETED" info: publicInfo: entitiesCommitted: 10 privateInfo: entitiesCommitted: 10 glossaryId: dfasd13 logs: - timestamp: "2021-09-30T10:00:00Z" level: "INFO" message: "Entity provisioned successfully" 400: description: Invalid Request content: application/json: schema: $ref: '#/components/schemas/RequestValidationError' examples: missingToken: value: errors: - "Token is missing" 500: description: Internal Error content: application/json: schema: $ref: '#/components/schemas/SystemError' examples: internalError: value: error: "Internal Error" userMessage: "An internal error occurred" moreInfo: problems: - "An internal error occurred" solutions: - "Please try again later" 501: description: Not implemented content: application/json: schema: $ref: '#/components/schemas/SystemError' examples: notImplemented: value: error: "Not implemented" userMessage: "This feature is not implemented yet" moreInfo: problems: - "This feature is not implemented yet" solutions: - "Please contact the platform team." /v1/entity/reference: get: tags: - DataCatalogPlugin summary: Get linked Data Catalog entity description: | Return the reference (id, links, etc) to the Data Catalog entity that refers to the provided Output Port operationId: getEntityReference parameters: - name: componentId in: query description: Output Port URN to get the reference to the Data Catalog entity required: true schema: type: string example: urn:dmb:dp:finance:a-system:0:component:1 responses: 200: description: Data Catalog Entity reference content: application/json: schema: $ref: '#/components/schemas/EntityReference' examples: entityReference: value: reference: "https://datacatalog.witboost.com/entity/finance.a-system.0.component.1" 500: description: Internal Error content: application/json: schema: $ref: '#/components/schemas/SystemError' examples: internalError: value: error: "Internal Error" userMessage: "An internal error occurred" moreInfo: problems: - "An internal error occurred" solutions: - "Please try again later" components: schemas: ValidationStatus: type: object required: - status properties: status: type: string enum: [ RUNNING, COMPLETED, FAILED ] description: > - `RUNNING`: the validation task is still running - `COMPLETED`: the operation is complete and the validation result is available (`info.publicInfo` field) - `FAILED`: an exception occurred while running the validation task info: $ref: "#/components/schemas/ValidationInfo" ValidationResult: description: Result of a validation operation on a provided descriptor required: - valid type: object properties: valid: description: Whether the provided descriptor is valid or not type: boolean error: $ref: '#/components/schemas/ValidationError' ValidationError: description: Validation error related to a provided descriptor required: - errors type: object properties: errors: type: array items: type: string DescriptorKind: type: string enum: [ DATAPRODUCT_DESCRIPTOR, DATAPRODUCT_DESCRIPTOR_WITH_RESULTS ] ProvisioningRequest: required: - descriptorKind - descriptor type: object properties: descriptorKind: $ref: '#/components/schemas/DescriptorKind' descriptor: type: string description: A provisioning request in yaml format ProvisioningStatus: required: - status - result properties: status: type: string enum: [ RUNNING, COMPLETED, FAILED ] result: deprecated: true type: string info: $ref: "#/components/schemas/Info" logs: type: array items: $ref: '#/components/schemas/Log' RequestValidationError: required: - errors type: object properties: errors: type: array deprecated: true items: type: string userMessage: type: string description: User-readable message to be displayed input: type: string description: Optional field to include the file or descriptor that raised the error inputErrorField: type: string description: Optional field to include the field path (in dot format) that raised the error moreInfo: $ref: "#/components/schemas/ErrorMoreInfo" SystemError: required: - error type: object properties: error: type: string deprecated: true userMessage: type: string description: User-readable message to be displayed input: type: string description: Optional field to include the file or descriptor that raised the error inputErrorField: type: string description: Optional field to include the field path (in dot format) that raised the error moreInfo: $ref: "#/components/schemas/ErrorMoreInfo" ErrorMoreInfo: required: - problems - solutions type: object description: Object that will include the more in-depth, specific information about the error properties: problems: type: array description: "Array of possible multiple problems: i.e. multiple validations failed" items: type: string solutions: type: array description: Array of possible solutions that the developer gives to the user to solve the issue items: type: string EntityReference: required: - reference properties: reference: type: string description: Reference to the Data Catalog entity. Either an ID or a link to the entity in the Data Catalog. Info: type: object required: - publicInfo - privateInfo properties: publicInfo: type: object description: | Fields to display in the Witboost Marketplace. Note that only the values compliant to specific structures will be rendered in the "Technical Information" card of the Marketplace pages. [Check the documentation](https://docs.internal.witboost.agilelab.it/docs/p2_arch/p6_other_modules/p6_4_dataCatalog.md) for additional details privateInfo: type: object description: "All the values in this object will be stored in the deployed descriptor, but will not be shown in the Witboost Marketplace" ValidationInfo: type: object required: - publicInfo properties: publicInfo: $ref: "#/components/schemas/ValidationResult" Log: type: object required: - timestamp - level - message properties: timestamp: type: string format: date-time level: description: This is the severity level of the log type: string enum: [ DEBUG, INFO, WARNING , ERROR ] message: type: string phase: type: string