Changes between Version 9 and Version 10 of ControlAPICliDesign

Aug 18, 2018, 12:20:44 AM (15 months ago)



  • ControlAPICliDesign

    v9 v10  
    11Note: this is work in progress. Please expect significant changes.
    3 = Control API text client =
     3Migrated to gitlab. Please make changes here:
    5 We have Control API interface [wiki:ControlAPIRequirements described] and [wiki:ControlAPIDesign designed], so it's time to design a client that could take advantage of that design.
    7 == Requirements ==
    9 Functional requirements:
    11 - F.1 - '''client MUST provide command line (text) interface'''. The major feature is the REST API itself. It is expected to be used by various IPAM management systems, so the text client is considered more of an extended proof of concept rather than a super feature rich client on its own.
    12 - F.2 - '''client MUST be scripting friendly'''. Nevertheless, the client will be used on its own. For small deployments there's no that big incentive to use Control API as the configuration could be updated and the server restarted. It seems the client will be mostly used by medium and large scale deployments. It is expected that such deployments already have their own systems and will use the client to reflect the changes, e.g. add new reservation for new student in a campus, remove leases and reservations for a subscriber that terminated his contract etc. This kind of changes will typically be done automatically.
    13 - F.3 - '''manual command execution MUST be possible'''. As explained in F.2 comment, the primary focus should be on scripting friendliness and automation. Nevertheless, it must be possible to manually run the client to send commands to Kea and have some sane way to present the response.
    14 - F.4 - '''interactive mode MAY be supported'''. If time permits, it should be able to support sending multiple commands and receive multiple responses in a loop. This may be useful for cases when there is lots of commands (e.g. many subscribers being added frequently). This is low priority, so it is unclear if/when this mode will be implemented.
    16 Maintenance requirements:
    18 1. M.1 - '''the client MUST NOT impose additional dependencies on all installations'''. While the concept of a Control API client is appealing to many, there's certainly a large group of users who do not intend to use it at all. As such, the code must be written in a way that will not incur any extra compilation complexity to those who don't need the client. In particular, the new code must not introduce additional dependencies.
    19 2. M.2 - '''the client SHOULD be extensible'''. This is essential for long term maintainability of the code. Right now the number of supported commands is modest, but we hope to grow that number over time. As such, it must be possible and easy to extend the client to support additional commands.
    21 == Design proposal 1: shell + curl ==
    22 One design proposal assumes that the script could be written as a shell script that takes heavy use of curl. Curl is a command line swiss army knife type of tool that supports http, https, ftp, tftp, ssl, tls, ipv4, ipv6 and lots of other protocols. It is available from, but it's commonly present in most distributions. It's licensed under MIT/X license.
    24 This approach assumes the major logic is implemented in a shell script that calls curl to take care of the http (or https) communication. To avoid requiring curl on all Kea deployments, the client tools would be disabled by default and would require explicit switch passed to configure (e.g. --enable-ctrl-client) to be enabled. Once enabled, it would require the curl to be installed in the system.
    26 This approach has several benefits and two major flaws:
    27 - PRO: It's easy to have something simple be implemented quickly. Within days we could have a crude, but working client.
    28 - CON: Adds extra dependency (curl required)
    29 - CON: It would be difficult to provide more complex functions as shell lacks the ability to generate or parse JSON syntax. It would be possible to parse JSON, but extra tools would be required.
    31 == Design proposal 2: python ==
    33 This design assumes the control API client will be written in python. We do have a long and sad history of using Python going back to BIND10 days. I like to think that BIND10 was harsh, but valuable lesson. In my opinion, two things were done incorrectly in BIND10: dependence on python3 version and making python mandatory. To avoid those issues, the client code must be disabled by default and only be enable with explicit command (see M.1 requirement). Also, the code must be able to run correctly in both 2.x and 3.x Python environments.
    35 Assuming those two conditions are followed, python provides a number of advantages:
    37 - there are several python libraries that can handle http(s) connection: urllib, httplib and possibly others. We need to pick up one that is available in both 2.x and 3.x. This link provides some insight:
    38 - there's json library available in both 2.x and 3.x. (There are also alternatives, e.g. simplejson, but they're not part of the standard python distribution, and thus induce additional dependencies. That's something we want to avoid)
    39 - python is a full featured language that provides many features we could use: json validation, complex parameter validation, function callbacks (could be useful for registering special processing for certain commands)
    41 Given the mitigation steps for drawbacks, multitude of benefits and some discussions on Jabber, it seems we should go with python, rather than with shell script.
    43 == Name ==
    44 The number of Kea components keeps growing. It would be good to set the proper name from the beginning. Since kea-admin, and keactrl are already taken and kea-client is reserved for DHCP client, I think we should go with '''kea-shell'''. This would be good for two reasons. First, it's somewhat similar to omshell in dhcpd, so the similarity would help people who are migrating. Second, the 'shell' part would make it obvious that it's a command line tool, not a gui.
    46 == Design ==
    48 kea-shell will be a command line tool written in python. The key point in understanding the approach here is that it is unknown what commands CA (and Kea servers behind CA) will be supporting. We intend to take advantage of the hooks mechanism and will encourage third parties to do the same. As such, we can't realistically claim that kea-shell will have a complete support for all commands. Therefore we need to have '''generic mode''' that would be able to handle any command as long as it follows the general syntax.
    50 At the same time, having dedicated support for commands that we know are there may be very useful. For example, when we know that the command is to retrieve a single statistic, it may be useful to print out the statistic value only rather than the whole JSON structure that says the status is ok and has other extra information. If there's a dedicated handler for a command, it would be called '''dedicated mode'''.
    52 kea-shell will be start by looking in its directory and loading all ks_*.py files (ks stands for kea-shell). Each of those files will have complete set of commands needed to handle a single command. In principle there will be two functions expected to be implemented:
    54 {{{
    55 #!python
    56 # This file contains code specific for handling Kea control agent command: foo
    58 # This method takes the input specified by the user and verifies it.
    59 # If the input is incorrect, it is expected to raise TBD
    60 # The input may be sanitized (e.g. some default values may be added) and the
    61 # full request should be returned. This will be returned as is to the Kea
    62 # control agent.
    63 #
    64 # The input or the value returned must not contain any HTTP headers. These
    65 # will be included by kea-shell that calls this method.
    66 #
    67 # If there is no validation done and input is used as is, this may be
    68 # implemented as:
    69 #
    70 # return (request_text)
    71 #
    72 # @param: request_txt input text specified by the user
    73 # @return: request to be sent to the Kea control agent
    74 def request_process(request_txt):
    76        ...
    77    return (validated_request)
    79 # This method teakes the reponse returned by the server.
    80 #
    81 # This method may do some processing of the content (e.g. extract the core
    82 # information and toss the unnecessary JSON syntax around it) and determine
    83 # the status of te request. Both should be returned.
    84 #
    85 # @param response response provided by the server
    86 # @return a pair of status-code (an integer to be returned as exit code) and
    87 #      processed response (that will be printed on stdout)
    88 def response_process(response):
    89    ...
    90    return (status-code, processed_response)
    91 }}}
    93 The kea-shell will take several optional parameters:
    94 - '''--host''' - which will specify the Kea control agent location. This could be either IP address or hostname. If not specified, "localhost" will be used.
    95 - '''--port''' - This will specify the TCP port to be used for Kea control agent connection. If not specified, the default value of 8081 will be used.
    96 - '''--legacy''' - This switch can be used to enforce using legacy IP protocol version 4
    97 - '''--generic|dedicated''' - This enforces one of the modes: generic or dedicated.
    98 - '''--timeout''' - Specifies the timeout before kea-shell gives up waiting for the CA response
    99 - '''--help''' - provide some guideance for lost users
    101 Kea-shell will also need an optional parameter that will specify the command, e.g:
    103 {{{
    104 kea-shell --host localhost --port 8081 list-commands
    105 }}}
    107 If command is specified, the tool will expect only command parameters on its stdin. For some parameter-less commands, there may be no input required at all.
    109 Here's the envisaged control flow:
    111 - check command line parameters (print error and help if the parameters provided don't make sense)
    112 - parse input provided on stdin. If it's not a valid JSON, print out generic error and exit
    113 - check if the input contains valid command name. If it doesn't, print an error and exit.
    114 - attempt to load ks-[name of the command].py. If the load is successful, the code will work in dedicated mode. If it fails (e.g. because the file is not there), the tool will work in generic mode. If dedicated flag is specified and the file is not there, abort. If the generic flag is specified, skip this step.
    115 - (in dedicated mode only): pass the input to request_process function and get validated_request in reponse. This will be sent to the server.
    116 - open HTTP connection to the server.
    117 - send the input, wait for response
    118 - (in dedicated mode only): pass the received output to response_process function and get the exit-code and processed_response
    119 - print out the response (or processed_response)
    120 - terminate process with return code indicating result (0 = successful, non-zero = failure)
    122 In the future, we may extend this tool with additional capabilities. However, given the time and manpower constraints we have, this should be sufficient to get a simple, but functional tool.