Introduction#

Structure of this Manual#

Besides this Introduction page, the manual also contains the following chapters:

  • The Developer Manual describes the tools made available to model and training data developers.

  • The Exploitation Manual explains how users may use the Marketplace to discover the available models and run these models to obtain predictions.

  • In Using the STAC Specification developers will find advices to create the STAC Item descriptors that represent the models and training data resources they want to share and publish.

High-Level Architecture#

The AIOPEN platform is an innovative, cloud-native environment designed to streamline the development, deployment and execution of AI models. The platform architecture is strategically divided into separate yet interconnected components that cater to both the development and execution phases, ensuring a seamless work flow from concept to production.

../_images/platform-overview.png

Platform Overview#

GitHub#

GitHub sits at the periphery of our architecture, offering a sophisticated version control system that not only safeguards the source code but also integrates with the platforms continuous integration pipeline. It provides a familiar interface for developers to manage code changes, review history and collaborate on code development. The platform works in a GitOps approach, whereby the infrastructure and application life-cycle is managed from a series of Git repositories as the single source of truth.

Development Environment#

The development environment is the crucible where ideas take shape and transform into tangible AI models. It works in tandem with the Workspace Controller, which is responsible for orchestrating workspaces and resources tailored to the needs of the development teams. Each workspace comes with it’s own Model Registry, Storage, local catalogue to store developmental STAC items and dedicated IDEs.

  • Model Registry: An instance of MLflow is created. It is a centralized repository that safely manages the versioning and storage of AI models, facilitating ease of access and collaboration among team members within the workspace.

  • Storage: A secure and scalable solution for handling the vast datasets required for AI training and validation. It provides dedicated S3 buckets to store, retrieve and manage data efficiently.

  • Local Catalogue: An instance of EOEPCA resource catalogue that can stores STAC item representations of Training Data and AI Models within the confines of the users workspace.

  • IDEs: A user can start and stop dedicated IDEs as part of a workspace.

Multiple workspaces can be instantiated, each becoming a silo for various project teams. The user has access to a development environment UI where they can configure and start their own IDE instances.

Development Services UI#

The user must login with their GitHub credentials to access their personalized UI. This is where they can configure their IDEs, SSH keys and request API keys to access their MLflow model registry externally from the platform. It has has a STAC validator service, allowing users to check if the STAC item’s they are to commit to git will be valid for insertion into the global or local catalogue, based on the schema defined within them.

Global Catalogue#

This is where end of development meets the start of exploitation. When a user commits a STAC item from the main branch or merges to main branch, it will be inserted into the Global Catalogue and be accessible in the marketplace.

Marketplace#

Users can view STAC items that were committed from their workspace repository as products on the marketplace.

Exploitation#

Users can exploit their models and training data and can run inference on their models through the dedicated website portal.

Becoming a User#

The AIOPEN platform offers two main categories of services:

  • Regular users may discover the trained models published on the Marketplace and select models for running predictions. Models may be deployed and executed on-demand, directly in the web interface, or started as services and called programmatically using a OGC API Processes interface.

  • In addition, Dataset and Model developers have access to services that allow them develop, train, and test their models directly inside the AIOPEN environment.

Currently, users and developers may not register themselves in AIOPEN. Please contact us at mailto:aiopen@spaceapplications.com should you want to learn more.