Configuration ============= **hcpcbc** uses a `JSON `_ file to read its configuration from. The loader does some basic consistency checks, but it's required to make sure that the configuration file strictly complies to the syntax rules for JSON files. It is highly suggested not to change the structure of the file. .. Warning:: Even simple errors in the JSON file might trigger errors like this:: Parsing the config file "hcpcbc_config.json" failed: hint: Expecting property name enclosed in double quotes: line 66 column 13 (char 2602) Luckily, the position given points fairly exact to where the error is - in this case, it's just a comma too much: .. code-block:: json :lineno-start: 58 :emphasize-lines: 9-10 }, "x": { "0 enable collection": true, "1 adminAllowed": false, "2 user": "", "3 password": "", "4 systemVisibleDescription": "", "9 last collected": "2015-12-30T20:36:25+0100" }, } A fresh configuration file configures access to the HCP system(s) using a system-level user, only. At the very beginning of each run, **hcpcbc** queries the HCP systems for a list of configured Tenants. At the end of a run, the configuration file is refreshed with this gained details. If there are Tenants where the user account used has no access to, a different (Tenant-level) user account can be configured for that specific Tenant by simply filling the provided (empty) fields with the users name and password. .. Warning:: As the configuration file contains user credentials, make sure that it is stored in a secure location with propper permission settings, **not to disclose it to nosy eyes!** The file must be write-able by the tool itself, as it uses it to remember state between its runs. The configuration file explained ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ This is the template configuration file **hcpcbc** is able to create for convenience, if no configuration file is available. .. code-block:: json :lineno-start: 1 { "0 comment": [ "This it the configuration file for **hcpcbc**. For a detailed", "description visit *https://hcpcbc.readthedocs.org*." ], **Target section** This section describes the HCP system(s) from which chargeback data is to be collected. Any number of HCP systems can be defined. | The **user** needs to be a system-level user having the *Monitor* role. | The **folder** is a sub-folder below the *path(s)* declared in the store sections. .. code-block:: json :emphasize-lines: 7-11,18-22 :lineno-start: 6 "1 targets": [ { "0 comment": [ "An HCP system to collect from. *User* needs to be a", "system-level user that has the *Monitor* role enabled." ], "1 fqdn": "hcp72.archivas.com", "2 user": "monitor", "3 password": "monitor01", "4 folder": "hcp72", "9 tenants": {} }, { "0 comment": [ "An HCP system to collect from. *User* needs to be a", "system-level user that has the *Monitor* role enabled." ], "1 fqdn": "hcp73.archivas.com", "2 user": "monitor", "3 password": "monitor01", "4 folder": "hcp73", "9 tenants": {} } ], **Parameter section** | The **periode** sets the granularity of the collected data. A list of one, two or three values (out of "hour", "day", "total") may be given. | **collection limit** declares how many days worth of data shall be gathered in a single run. If **hcpcbc** fails with timeout errors, lower the value from the default of 181 days down to some ten days. Then run **hcpcbc** multiple times until the collection has reached the actual date. | **timeout** defines the TCP connection timeout in seconds. .. code-block:: json :emphasize-lines: 8-11,13,14 :lineno-start: 30 "2 parameters": { "0 comment": [ "*1 periode* defines the reports granularity,", "*2 collection limit* defines for how many days collection", "takes place, starting from the last collection time." "*3 timeout* defines the tcp connection timeout" ], "1 periode": [ "total", "day", "hour" ], "2 collection limit": 181, "3 timeout": 600 }, **Stores section** In this section, storage locations are defined for: * temporary storage - this will be used to store reports during processing * local storage - a local directory in which the collected reports will be stored * compliant storage - an HCP Namespace to store the reports, if required with a retention periode set * **path** - a full qualified path to the Namespace * **user** - needs to have write access to the Namespace * **retention** - see HCP manual "Using a Namespace" for valid retention patterns * **store raw** - store the raw reports (beside the split-up reports for Tenant summary and Namespaces) .. code-block:: json :emphasize-lines: 2,7,9,14-16,18,23-28 :lineno-start: 45 "3 stores": { "1 temporary": { "0 comment": [ "The temporary directory in which the downloaded", "reports are stored during processing." ], "1 tempdir": "./_hcpcbc.dir" }, "2 local": { "0 comment": [ "The directory in which processed reports are", "stored locally." ], "1 enable": true, "2 path": "./_hcpcbc.dir", "9 store raw": true }, "3 compliant": { "0 comment": [ "This is the HCP Namespace to which the processed reports", "can be stored for long-term preservation" ], "1 enable": false, "2 path": "https://n1.m.hcp72.archivas.com/rest/chargeback", "3 user": "n", "4 password": "n01", "5 retention": "0", "9 store raw": true } }, **Charts section** This section defines which charts shall be created, if at all. * **path** can be the same as for *local store*; report and chart folder structure are made to allow overlap without interference. * **linear** and **log** enable charts with linear and/or logarithmic scale. .. code-block:: json :emphasize-lines: 7-12 :lineno-start: 75 "4 charts": { "0 comment": [ "Chart creation", "*hourly charts* will use the hour reports to create charts", "*daily charts* will do the same using the day reports." ], "1 enable": true, "2 path": "./_hcpcbc.charts", "3 hourly charts": true, "6 daily charts": true, "a linear": true, "b log": true }, **Logging section** This section defines how logging shall take place. .. code-block:: json :emphasize-lines: 5-10 :lineno-start: 88 "9 logging": { "0 comment": [ "This defines how hcpcbc does logging." ], "1 log to stdout": true, "2 log to file": false, "3 logfile": "./_hcpcbc.dir/_hcpcbc.log", "4 rotate MB": 10, "5 backups": 9, "6 debug": false } } Example configuration file after first run ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ After a run, the configuration file is updated with the Tenants configured per HCP system. .. code-block:: json :lineno-start: 1 { "0 comment": [ "This it the configuration file for **hcpcbc**. For a detailed", "description visit *https://hcpcbc.readthedocs.org*." ], "1 targets": [ { "0 comment": [ "An HCP system to collect from. *User* needs to be a", "system-level user that has the *Monitor* role enabled." ], "1 fqdn": "hcp72.archivas.com", "2 user": "monitor", "3 password": "monitor01", "4 folder": "hcp72", "9 tenants": { **Tenant section** In this section, **hcpcbc** stores a list of all Tenants configured in HCP. * **enable collection** - if set to "false", the Tenant will not be collected. This is usefull if you either don't have access to the Tenant *or* you don't want to collect it as it's a never-accessed replica, for example. * **adminAllowed** - if "true", system-level users has been granted access to Tenant management, so that no Tenant-level user is needed to access Namespace-level reports. * **user**, **password** - not required if **adminAllowed** = **true**; otherwise, if manually added, will allow to collect Namespace-level reports. * **systemVisibleDescription** - for convenience * **last collected** - the timestamp of the last collection from the Tenant .. code-block:: json :emphasize-lines: 1-7 :lineno-start: 17 "m": { "0 enable collection": true, "1 adminAllowed": true, "2 user": "", "3 password": "", "4 systemVisibleDescription": "Der ueblicherweise als erstes erzeugte Tenant...", "9 last collected": "2015-12-31T06:49:16+0100" }, "s3": { "0 enable collection": true, "1 adminAllowed": true, "2 user": "", "3 password": "", "4 systemVisibleDescription": "", "9 last collected": "2015-12-31T06:49:26+0100" }, "s3erlei": { "0 enable collection": true, "1 adminAllowed": true, "2 user": "", "3 password": "", "4 systemVisibleDescription": "", "9 last collected": "2015-12-31T06:48:59+0100" }, "swifttest": { "0 enable collection": true, "1 adminAllowed": true, "2 user": "", "3 password": "", "4 systemVisibleDescription": "", "9 last collected": "2015-12-31T06:48:53+0100" }, "swifty": { "0 enable collection": true, "1 adminAllowed": true, "2 user": "", "3 password": "", "4 systemVisibleDescription": "", "9 last collected": "2015-12-31T06:49:06+0100" }, "testtenant": { "0 enable collection": true, "1 adminAllowed": false, "2 user": "", "3 password": "", "4 systemVisibleDescription": "", "9 last collected": "2015-12-31T06:49:22+0100" }, "x": { "0 enable collection": true, "1 adminAllowed": false, "2 user": "", "3 password": "", "4 systemVisibleDescription": "", "9 last collected": "2015-12-31T06:49:03+0100" } } }, { "0 comment": [ "An HCP system to collect from. *User* needs to be a", "system-level user that has the *Monitor* role enabled." ], "1 fqdn": "hcp73.archivas.com", "2 user": "monitor", "3 password": "monitor01", "4 folder": "hcp73", "9 tenants": { "aw21": { "0 enable collection": true, "1 adminAllowed": true, "2 user": "", "3 password": "", "4 systemVisibleDescription": "", "9 last collected": "2015-12-31T06:49:58+0100" }, "m": { "0 enable collection": true, "1 adminAllowed": true, "2 user": "", "3 password": "", "4 systemVisibleDescription": "Der ueblicherweise als erstes erzeugte Tenant...", "9 last collected": "2015-12-31T06:50:24+0100" }, "s3": { "0 enable collection": true, "1 adminAllowed": true, "2 user": "", "3 password": "", "4 systemVisibleDescription": "", "9 last collected": "2015-12-31T06:49:54+0100" }, "s3erlei": { "0 enable collection": true, "1 adminAllowed": true, "2 user": "", "3 password": "", "4 systemVisibleDescription": "", "9 last collected": "2015-12-31T06:49:45+0100" }, "swifttest": { "0 enable collection": true, "1 adminAllowed": true, "2 user": "", "3 password": "", "4 systemVisibleDescription": "", "9 last collected": "2015-12-31T06:50:11+0100" } } } ], "2 parameters": { "0 comment": [ "*1 periode* defines the reports granularity,", "*2 collection limit* defines for how many days collection", "takes place, starting from the last collection time.", "*3 timeout* defines the tcp connection timeout" ], "1 periode": [ "total", "day", "hour" ], "2 collection limit": 181, "3 timeout": 600 }, "3 stores": { "1 temporary": { "0 comment": [ "The temporary directory in which the downloaded", "reports are stored during processing." ], "1 tempdir": "./_hcpcbc.dir" }, "2 local": { "0 comment": [ "The directory in which processed reports are", "stored locally." ], "1 enable": true, "2 path": "./_hcpcbc.dir", "9 store raw": true }, "3 compliant": { "0 comment": [ "This is the HCP Namespace to which the processed reports", "can be stored for long-term preservation" ], "1 enable": false, "2 path": "https://n1.m.hcp72.archivas.com/rest/chargeback", "3 user": "n", "4 password": "n01", "5 retention": "0", "9 store raw": true } }, "4 charts": { "0 comment": [ "Chart creation", "*hourly charts* will use the hour reports to create charts", "*daily charts* will do the same using the day reports." ], "1 enable": true, "2 path": "./_hcpcbc.charts", "3 hourly charts": true, "6 daily charts": true, "a linear": true, "b log": true }, "9 logging": { "0 comment": [ "This defines how hcpcbc does logging." ], "1 log to stdout": true, "2 log to file": false, "3 logfile": "./_hcpcbc.dir/_hcpcbc.log", "4 rotate MB": 10, "5 backups": 9, "6 debug": false } }