<!-- HEADER -->
<a name="readme-top"></a>


<!-- PROJECT -->
<br />
<div align="center">
<h3 align="center">NGCP Task Agent</h3>
</div>


<!-- TABLE OF CONTENTS -->
<details>
  <summary>Table of Contents</summary>
  <ol>
    <li>
      <a href="#about-the-project">About The Project</a>
      <ul>
        <li><a href="#built-with">Built With</a></li>
      </ul>
    </li>
    <li>
      <a href="#getting-started">Getting Started</a>
      <ul>
        <li><a href="#installation">Installation</a></li>
        <ul>
          <li><a href="#dependencies">Dependencies</a></li>
        </ul>
        <li><a href="#concept">Concept</a></li>
        <li><a href="#usage">Usage</a></li>
      </ul>
    </li>
    <li><a href="#license">License</a></li>
    <li><a href="#contact">Contact</a></li>
  </ol>
</details>


<!-- ABOUT THE PROJECT -->
## About

This application is an asynchronous task/message queue server that helps with data delivery,remote command execution and various of others async tasks that needs to be performed automatically or on demand inside NGCP powered platforms.


<!-- BUILT WITH -->
### Built With

* [![python3][python3]][pyton3-url]
* [![python3-asyncio][python3-asyncio]][python3-asyncio-url]


<!-- GETTING STARTED -->
## Getting Started


<!-- INSTALLATION -->
### Installation

The installation can be done by installing the ngcp-task-agent package via apt

```
apt install ngcp-task-agent
```

<!-- DEPENDENCIES -->
#### Dependencies

```
python3-ngcp-task-agent
```

```
python3
python3-aioredis
python3-setproctitle
python3-yaml
```

<!-- CONCEPT -->
### Concept

#### The App (Server)

The app part is run as ngcp-task-agent service and invoked as

```
/usr/sbin/ngcp-task-agent
```

#### Config

All config files reside inside ``/etc/ngcp-task-agent``

```
/etc/ngcp-task-agent
/etc/ngcp-task-agent/backends
/etc/ngcp-task-agent/backends/redis.yml
/etc/ngcp-task-agent/tasks
/etc/ngcp-task-agent/tasks/clear_call_counters.yml
```

All the config files are in YAML format and end (expected to be) with the .yml extension


#### Backend

`backend` is the app's core component that is responsible for receiving and delivering messages from/to clients. For instance the available `redis` backend connects to the Redis server and waits for incoming messages from clients.


#### Broker

`broker` is another app's core component that upon receiving a task provides with the execution of the task and returns responses based on when a task is being accepted and processed, error messages or completion messages. For example the available `command` broker when receiving a task, executes a script with arguments provided in the task.


#### Task

`task` is the component that in terms of ngcp-task-agent acts as a dispatcher between the `backend` and the `broker`. A `task` contains no backend/broker specific logic. When a task is created by the app it always consists of at least 2 required parts, the `backend+backend options` and the `broker+broker options`. A task creates callbacks to the backend (attaches to it) and to the broker so when a message is received by the backend it is provided to the task as the callback. When a message is received by a task, it controls if the message is actually meant to be processed or otherwise rejects it (wrong destination, non-existing task/backend/options, etc.), otherwise it delivers the task to the related broker and delivers messages from the broker back to the backend.


#### Messages

A JSON message that is used to deliver request or responses from to clients.


##### Request

A message that is delivered from a client to the server (backend).

```
uuid (str): unique id of the request (UUID),
  must be provided by the client
task (str): task name, if non-existent,
  rejected by the server
src (str): source host name
  (or another known to the server identifier)
dst (str): destination host name
  (or can be a wildcard [fnmatch] match) that in case of multiple task agent
  instances to only process the request on some of them.
  it also supports inline options, e.g.:
  dst='local' - accept the request on the local node
  dst='localhost' - accept the request on the local node
  dst='prx01a' - accept the request on prx01a node
  dst='prx*' - accept the request on all proxy nodes
  dst='*|state=active' - accept the request on all active nodes'
  dst='*|status=online'
    accept the request on all nodes with online status
  dst='*|state=active;role=proxy' - accept the request on all active proxy nodes'
  dst='*|state=active;role=proxy+lb' - accept the request on all active proxy and lb nodes'
  dst='prx01a,lb*,db01?|state=active' - accept the request on prx01a, all lb nodes
    and the active db01 node
timestamp (float): unixtime of the message,
  generated by the server upon receiving the request
datetime (str): datetime of the message,
  generated by the server upon receiving the request
options (Dict): task related options,
  usually something that is expected
  by the backend, like 'feedback_channel'
data (Any): additional data to provide to the server
  (ignored by the server if contains unexpected data)
```


##### Response

A message that is provided from the server (backend) to a client.

```
uuid (str): unique id of the response
ref (str): unique id of the original request,
  used to relate responses to the request
  task (str): task name that is used in the response
  (usually the same that is provided by the request)
src (str): server host name
dst (str): originated client host name
timestamp (float): unixtime of the message,
  generated by the server when dispatching the response
datetime (str): datetime of the message,
  generated by the server when dispatching the response
chunk (int): chunk number of the response (default: 1),
  in cases when a response is split into multiple chunks
  each next response's value is increased by 1
chunks (int): (default: 1) in case if a response is split by the server
  into multiple chunks, contains the total amount of chunks
status (str): [accepted|rejected|done|error]
  indication of the server state related to the response,
  (accepted): response is sent by the server if the request is accepted
    and just before processing it by the broker
  (rejected):  response is issued if something went wrong,
    and usually the 'reason' field contains relevant info about it
  (done): when a task is completed and in this case the response
    contains the final data
    (in case of chunked responses they all have the same 'status')
  (error): when an error occurred on the server side
reason (str): reason string,
  empty in case of 'accepted' or 'done'
  and used for 'rejected' and 'error' responses
data (Any): - resulting data that is provided by the server
```


<!-- USAGE -->
## Usage


### Backend configuration

Backends may or may not contain specific configurations, in case if they do, the config files reside in `/etc/ngcp-task-agent/backends/`

Example of the `redis backend configuration`:

`/etc/ngcp-task-agent/backends/redis.yml`

```
---
host: 127.0.0.1
port: 6379
db: 40
```


### 'redis' backend options

`control_channel` - when a task that uses the redis backend starts the backend uses the control_channel option to subscribe to the channel and listen for incoming messages.


### 'command' broker options

`run` - a script name to run
`as_user` - run the script as a user
`args` - command arguments


### Task configuration

A new task can be created by simply putting one into `/etc/ngcp-task-agent/tasks/`

For example our goal is to ping address 10.10.10.10 on the server that runs ngcp-task-agent.

First the task needs to be created and configured:

`/etc/ngcp-task-agent/tasks/ping_10_10_10_10.yml`

```
---
name: ping_10_10_10_10
backend: redis
backend_options:
    control_channel: ngcp-task-agent-redis
broker: 'command'
broker_options:
    run: ping
    as_user: root
    args: -c 2 10.10.10.10
```

### Client side invocation

On the client side in case of the `redis` backend the client must perform the following steps:

1. connect to Redis and subscribe to a unique channel that will be used by the server for the feedback (called in this case as `feedback_channel`)
2. publish to Redis a request to the same channel that is `control_channel` in the task's config
3. receive responses from the server in the `feedback_channel`

Example of `ping_10_10_10_10` task request

```
{
 'uuid': 'cb88ac76-f348-4614-866f-43877b893e7e',
 'task': 'ping_10_10_10_10',
 'src': 'sp1',
 'dst': 'sp1',
 'options': {'feedback_channel': 'fb_ctrl_7056'},
 'data': null
}
```


<!-- LICENSE -->
## License

[![GPL3 License][license-gpl3]][license-url]
Distributed under the GPL3 License. For more information check `LICENSE.txt`


<!-- CONTACT -->
## Contact

Sipwise Development Team - (https://www.sipwise.com) - support@sipwise.com

Project Link: [https://github.com/sipwise/ngcp-task-agent](https://github.com/sipwise/ngcp-task-agent)

[license-gpl3]: https://img.shields.io/badge/license-gpl3-red
[license-url]: https://www.gnu.org/licenses/gpl-3.0.en.html
[python3]: https://img.shields.io/badge/python3-blue
[pyton3-url]: https://python.org/
[python3-asyncio]: https://img.shields.io/badge/python3-asyncio-blue
[python3-asyncio-url]: https://docs.python.org/3/library/asyncio.html
