wiki:DbConfigStorageDesign

Kea Configuration Backend Design

Migrated to Gitlab. Please see: https://gitlab.isc.org/isc-projects/kea/wikis/designs/configuration-in-db-design


This is a design document for the Kea Configuration Backend feature in Kea. This feature is planned to be included in the Kea 1.5.0 release.

Introduction

This Kea Configuration Backend is a new feature of Kea which allows for storing configuration of Kea server in a database. Parts of the configurations can still be provided in the configuration file and such local configuration takes precedence over the configuration provided in the database. The configuration file may now include connection information and credentials to be used by the server to connect to the database and fetch the configuration information. Additional control commands and necessary updates to the existing control commands are going to be introduced to allow for managing any piece of the server configuration that can be stored in the database. The database schema contains ancillary data for the servers to be determine when new changes to the configuration have been introduced, e.g. subnet removed, subnet added, subnet modified etc.

Design Considerations

The following is a list of various aspects of the design to be considered and debated:

  1. Database schemas and how that applies to various database backends
  2. Database content versioning
  3. Database credentials management
  4. Identification of Kea deamons, including DHCP servers within a single cluster and within the whole system
  5. Determine when the database changes should/can be fetched by respective daemons
  6. Address slight differences between configurations of certain servers, within the same shard
  7. Configuration daemon that can directly talk to the database
  8. Reuse of hooks libraries, e.g. subnet_cmds, to directly talk to the database
  9. Create new configuration in a db from scratch using a daemon
  10. Dump configuration from the database, by converting it to JSON
  11. Adaptation of the config-get, config-write etc. commands
  12. Provide central information about the DHCP servers in the database
  13. Isolation of the config backend code from the libdhcpsrv, perhaps even a hook?
  14. Possible extensions to kea-shell to wrap complex RESTful API commands into atomic operations
  15. How multiple DHCP servers using the same database configuration can work in isolation, e.g. can they assign addresses from the same subnet or different ones?

High Level Architecture

This picture shows three Kea instances, of which two are connected to the same database, i.e. DB1, and one is connected to the second database DB2. The Kea instances can comprise a DHCPv4 server, DHCPv6 server, Control Agent and/or D2 server. Each of these servers is capable of connecting to the database to fetch configuration. Also, each server uses a configuration file, which may comprise connection information and credentials required to connect to the database. In addition, the configuration file may contain configuration specific to the particular server instance. If this configuration overlaps with a configuration in the database for this server, the configuration from the file takes precedence over the configuration in the database. For example: if global DHCP parameters such as DHCP timers are specified in the JSON file and different values of these parameters are specified in the database, the values from the JSON file will be used.

In the Kea 1.5.0 release, only limited part of the server configuration can be stored in the database, i.e. subnets, networks, options, global parameters. The remaining parts of the DHCP server configuration must be provided in the JSON file. In Kea 1.5.0 release it will neither be possible to store Control Agent configuration nor D2 configuration in the database. Therefore, these daemons will have to be configured as usual, i.e. using the config files.

The Kea4 instance is used to provide a management interface to the entire system. It receives commands over the RESTful API and communicates directly with the database instances to insert new configurations, fetch configuration and present to an administrator or update the configuration. The process that is used to manage the database content is one or more of the existing Kea processes. The following cases are planned for the Kea 1.5.0 release:

  • A DHCPv4 server which doesn't respond to DHCP traffic, with suitable hooks libraries loaded,
  • A DHCPv6 server which doesn't respond to DHCP traffic, with suitable hooks libraries loaded,
  • A Control Agent with suitable hooks libraries loaded.

Obviously, allowing the Control Agent to use DHCP specific hooks libraries, e.g. subnet_cmds, will require some changes to those libraries to adopt them to be used by a different process. In particular, the Control Agent is not aware of the DHCP Configuration Manager which holds the DHCP configuration. Therefore, supporting the first two cases is mandatory. Supporting the third case is a stretch goal.

It is envisaged that some deployments may require additional configuration step which we call bootstrapping. One possible use case is when additional configuration data is held in yet another database instance. This additional configuration data may possibly affect how the rest of the configuration is performed. Imagine the case when the connection information to the DB1 and DB2 is held in a central database. The Kea server in the bootstrapping step could get the connection information from the central database and then use it to connect to the DB1 or DB2 to get the actual configuration information.

Since bootstrapping is highly dependent on the particular deployment, the bootstrapping code must be implemented as proprietary hooks library (custom hook). The Kea servers will be extended to provide new bootstrapping hook points. They will be called: dhcp4_srv_bootstrap and dhcp6_srv_bootstrap for the DHCPv4 and the DHCPv6 servers respectively. The names of the corresponding hook points for D2 and the CA process are to be determined.

In order to manage the content of the database use for bootstrapping there is a need to develop some custom management tool. Since such tool is proprietary it is out of scope for this document. However, it is worth noting that such tool can be integrated with Kea if it is implemented as a hooks library providing specific REST commands. This hooks library can be loaded by the Kea4 instance which in our example would be used to manage both the content of the DB1/DB2 and CustomDB1 databases.

Configuration Management

Management Only Configuration

{
    "command": "subnet4-add",
    "service": "ca",
    "arguments": {
        "database": {
            "type": "mysql",
            "name": "kea",
            "user": "kea",
            "password": "kea",
            "host": "marine.example.org"
        },
        "subnets": [ {
            "id": 123,
            "subnet": "10.20.30.0/24",
            ...
        } ]
    }
}

Manage and Fetch Configuration

{
    "command": "subnet4-add",
    "service": "ca",
    "arguments": {
        "database": {
            "type": "mysql",
            "name": "kea",
            "user": "kea",
            "password": "kea",
            "host": "marine.example.org"
        },
        "subnets": [ {
            "id": 123,
            "subnet": "10.20.30.0/24",
            ...
        } ]
    }
}

Manage and Fetch Configuration with DHCP Servers

{
    "command": "subnet4-add",
    "service": "dhcp4",
    "arguments": {
        "database": {
            "type": "mysql",
            "name": "kea",
            "user": "kea",
            "password": "kea",
            "host": "marine.example.org"
        },
        "subnets": [ {
            "id": 123,
            "subnet": "10.20.30.0/24",
            ...
        } ]
    }
}

Configuration Sequence

Configuration State Diagram

The server startup and configuration must be modified to introduce two new configuration phases required by the feature:

  • bootstrapping
  • server configuration fetching from a database

The following state diagram presents the updated startup and configuration phase.

The server starts as usual by reading the configuration file (ConfigFileRead). Bootstrapping requires that the hooks libraries implementing it are loaded. Therefore, the part of the configuration read from the file which contains a list of hooks libraries must be parsed and the hooks libraries must be loaded (NewHooksLibrariesLoad). If the libraries are loaded successfully, the server checks if there are any callouts implementing bootstrap hook points. The callouts are invoked (HooksSrvBootstrap) and can return one of the three status codes:

  • NEXT_STEP_CONTINUE - in which the Kea server will continue normally, i.e. will process configuration read from the file,
  • NEXT_STEP_SKIP - in which case the Kea server will skip loading configuration read from the file and will rather fetch configuration from the database, if the database connection information is provided,
  • NEXT_STEP_DROP - in which case the server will cease configuration and will rollback to the previous configuration (config error).

The server processes the configuration from the file (StagingConfig) where it presumably finds the connection information to the database. Note that such connection information could also be provided in the bootstrap step by the hooks library. The server will next connect to the database, fetch the configuration (DatabaseConfigFetch) and merge it into the existing (staging) configuration. Finally, the configuration is applied, i.e. logger is configured, D2 process is started, DHCP sockets are opened etc. (ConfigApply).

Bootstrap Hook

int dhcp4_srv_bootstrap(CalloutHandle& handle) {
    // Staging server configuration.
    isc::dhcp::SrvConfigPtr server_config;
    handle.getArgument("server_config", server_config);

    // Configuration from the config file.
    isc::config::ConstElementPtr config_file_contents;
    handle.getArgument("json_config", config_file_contents);

    // ...
    // Connect to another source of configuration information
    //  and apply the configuration to the staging configuration
    // object provided as "server_config". One possibility is to
    // set the configuration backend connection information and
    // credentials. The details how to do it is TBD.

    // Set the 'skip' status if there is no need for applying the configuration
    // read from the file (e.g. if the required configuration was fetched from
    // another source).
    handle.setStatus(CalloutHandle::NEXT_STEP_SKIP);
    return (0);
}

Database Schema

MySQL

TBD

Postgres

Postgres schema is going to be almost the same as MySQL database schema.

Cassandra

The following picture shows new tables to be included in the Kea CQL schema to facilitate storing DHCPv4 configuration in the database. The corresponding tables for DHCPv6 server will be also added to the design document. However, the DHCPv6 tables for options, networks and global parameters will be very similar to those for DHCPv4 case. There will be differences between the subnet4 and subnet6 tables being a result of slightly different way of defining pools. But the general concept of denormalization and storing hierarchical data within a single table applies.

Configuration Backend Structure

Class Diagram

The Configuration Backend feature must be structured to satisfy several constraints:

  • respective Kea servers and associated management hooks libraries should not need to include the unrelated code, e.g. CA should not need include/link with libkea-dhcpsrv to use the configuration backend feature
  • specific implementations of the Configuration Backend, e.g. MySQL, should be best isolated into separate modules so as they can be used optionally.

The following diagram segregates parts of the Configuration Backend system into separate packages/libraries. We may decide that such fine grained split is not desired for Kea 1.5.0 release, but this split is considered to give the most flexibility.

The ConfigBackendManager has a role similar to the LeaseMgr and HostMgr. It registers objects deriving from ConfigDataSource, which represent implementations of configuration data sources. Individual servers will create instances of specific data sources, e.g. configuration data source for D2, configuration data source for DHCPv4 server etc. Some modules, management hooks libraries may register more than one data source. For example, subnet_cmds hooks library may register objects deriving from !ConfigDataSourceDHCPv4 and !ConfigDataSourceDHCPv6 to manage both DHCPv4 and DHCPv6 specific configuration information.

Classes such as !ConfigDataSourceDHCPv4 and !ConfigDataSourceDHCPv6 are abstract interfaces. Implementations of these interfaces for various database flavors are best put into separate modules, e.g. libkea-cb-mysql, which would hold implementation of the configuration backend for MySQL. In fact, these libraries could be implemented as hooks and register themselves in the ConfigBackend manager when they are loaded.

Notes

Andrei suggested:

  • mention explicit indexes in DB schema.
  • mention primary keys
Last modified 10 months ago Last modified on Aug 17, 2018, 11:27:06 PM

Attachments (9)

Download all attachments as: .zip