Skip to content

title: Solver Atlas

Solver Atlas

Solver Atlas Gateway definition and implementation

  • Gateway describes a load balancer operating at the edge of the mesh receiving incoming or outgoing HTTP/TCP connections. Solver Atlas is built with the Istio object - Gateway, where we could also terminate SSL traffic.

Istio Mesh

Solver Atlas Auth

  • We are using Keycloak IDP with Istio's RequestAuthentication, but with an abstraction layer of our own Auth API to provide:

    • Single-Sign-On and Single-Sign Out for browser applications
    • OpenID Connect support
    • OAuth 2.0 support
    • SAML support
    • User Federation - Sync users from LDAP and Active Directory servers
    • Login flows - optional user self-registration, recover password, verify email, require password update, etc.
    • Session management - Admins and users themselves can view and manage user sessions.
    • Token mappers - Map user attributes, roles, etc. how you want into tokens and statements.

Solver Atlas Audit

  • Solver Atlas Audit is using Keycloak in conjunction with the Istio Request Authentication object for JSON Web Token proper consumption. For auditing, we are going to use a custom Envoy Filter, written in Lua language, that is going to collect audit trails and send them to the Audit API

Security

  • The Istio security features provide strong identity, powerful policy, transparent TLS encryption, and authentication, authorization, and audit (AAA) tools to protect services and data. The goals of Istio security are:

  • The configuration API server distributes to the proxies:

    • authentication policies
    • authorization policies
    • secure naming information

  • Sidecar and perimeter proxies work as Policy Enforcement Points (PEPs) to secure communication between clients and servers.

  • A set of Envoy proxy extensions to manage telemetry and auditing

Policies diagram

Solver Atlas User and Access Management

  • Users authenticate with Keycloak rather than individual applications. This means that your applications don't have to deal with login forms, authenticating users, and storing users. Once logged in to Keycloak, users don't have to log in again to access a different application.
  • This also applied to logout. Keycloak provides single-sign out, which means users only have to log out once to be logged out of all applications that use Keycloak. Keycloak has built-in support to connect to existing LDAP or Active Directory servers. Keycloak is based on standard protocols and provides support for OpenID Connect, OAuth 2.0, and SAML.

Solver Atlas Monitoring

  • We are going to use Grafana, Prometheus, Node Exporter, etc.. to create a dashboard for monitoring.
  • Hardware monitoring will look like this:

Atlas monitoring 1 Atlas monitoring 2

Solver Atlas Alerting

  • Alerting will be implemented through Grafana. With it, we will be able to implement notifications to Email, Slack, Teams, etc.

Solver Atlas IAM

  • We are using the abstraction of IAM API and Frontend APP to provide seamless integration with the complex RBAC system (roles and policies) of Istio AuthorizationPolicy and to allow privileged users to determine in-app Allow/Deny resources for other users.

Solver Atlas Predict API

  • This module is used for deploying and serving machine learning models in real time. The users can "register" (upload) a trained ML model (trained either through the platform or externally, i.e. within Jupyter Notebook) with the corresponding meta information (obtained from the documentation), and use it through the API to get model predictions.

CI/CD on Kubernetes

  • CI/CD process is going to be implemented via Jenkins which will build and push docker images and ArgoCD which is going to monitor the git repository of Kubernetes scripts. 

Solver Atlas - ETL Scheduler

  • Workflow - Solver Atlas

    • To improve ETL processes we are going to use Argo Events and Argo Workflow which is able to schedule in a number of different ways. We can implement parallel workflows as well.
    • Schedules are defined using the YAML format and provide the possibility of multiple dependencies.
    • The ETL process should look like the next picture which is used for visual validation.

Argo Workflow

Developers Hub

  • Developers Hub will contain a JupyterHub with Python kernel and other Python libraries such as Pandas, Numpy, SQLAlchemy, FastAPI, Flask, sklearn, etc.
  • User access and management will be provided through Solver Atlas User and Access Management.
  • JupyterHub will have access to Data Lake
  • JupyterHub will be integrated with Model Management, providing capabilities to train the model and assign into Model Management. Also, the same model can be called through JupyterHub.
  • JupyterHub is a playground with all functionalities for development of machine learning and data management models.
Back to top