Project: TDengine (App Definition and Development / Database)
File: 04-taosadapter.md

title: taosAdapter sidebar_label: taosAdapter description: This document describes how to use taosAdapter, a TDengine companion tool that acts as a bridge and adapter between TDengine clusters and applications. import Prometheus from "./_prometheus.mdx" import CollectD from "./_collectd.mdx" import StatsD from "./_statsd.mdx" import Icinga2 from "./_icinga2.mdx" import TCollector from "./_tcollector.mdx" taosAdapter is a TDengine companion tool that acts as a bridge and adapter between TDengine clusters and applications. It provides an easy-to-use and efficient way to ingest data directly from data collection agent software such as Telegraf, StatsD, collectd, etc. It also provides an InfluxDB/OpenTSDB compatible data ingestion interface that allows InfluxDB/OpenTSDB applications to be seamlessly ported to TDengine. taosAdapter provides the following features. RESTful interface InfluxDB v1 compliant write interface OpenTSDB JSON and telnet format writes compatible Seamless connection to Telegraf Seamless connection to collectd Seamless connection to StatsD Supports Prometheus remoteread and remotewrite Get table's VGroup ID If you use the TDengine server, you don't need additional steps to install taosAdapter. If you need to deploy taosAdapter separately on another server other than the TDengine server, you should install the full TDengine server package on that server to install taosAdapter. If you need to build taosAdapter from source code, you can refer to the documentation. On Linux systems, the taosAdapter service is managed by `systemd` by default. You can use the command `systemctl start taosadapter` to start the taosAdapter service and use the command `systemctl stop taosadapter` to stop the taosAdapter service. Use the command `rmtaos` to remove the TDengine server software if you use tar.gz package. If you installed using a .deb or .rpm package, use the corresponding command, for your package manager, like apt or rpm to remove the TDengine server, including taosAdapter. taosAdapter and TDengine server need to use the same version. Please upgrade the taosAdapter by upgrading the TDengine server. You need to upgrade the taosAdapter deployed separately from TDengine server by upgrading the TDengine server on the deployed server. taosAdapter is configurable via command-line arguments, environment variables and configuration files. The default configuration file is /etc/taos/taosadapter.toml on Linux. Command-line arguments take precedence over environment variables over configuration files. The command-line usage is arg=val, e.g., taosadapter -p=30000 --debug=true. The detailed list is as follows: ```shell Usage of taosAdapter: --collectd.db string collectd db name. Env "TAOSADAPTERCOLLECTD_DB" (default "collectd") --collectd.enable enable collectd. Env "TAOSADAPTERCOLLECTD_ENABLE" (default true) --collectd.password string collectd password. Env "TAOSADAPTERCOLLECTD_PASSWORD" (default "taosdata") --collectd.port int collectd server port. Env "TAOSADAPTERCOLLECTD_PORT" (default 6045) --collectd.ttl int collectd data ttl. Env "TAOSADAPTERCOLLECTD_TTL" --collectd.user string collectd user. Env "TAOSADAPTERCOLLECTD_USER" (default "root") --collectd.worker int collectd write worker. Env "TAOSADAPTERCOLLECTD_WORKER" (default 10) -c, --config string config path default /etc/taos/taosadapter.toml --cors.allowAllOrigins cors allow all origins. Env "TAOSADAPTERCORSALLOWALL_ORIGINS" (default true) --cors.allowCredentials cors allow credentials. Env "TAOSADAPTERCORSALLOWCredentials" --cors.allowHeaders stringArray cors allow HEADERS. Env "TAOSADAPTERALLOW_HEADERS" --cors.allowOrigins stringArray cors allow origins. Env "TAOSADAPTERALLOW_ORIGINS" --cors.allowWebSockets cors allow WebSockets. Env "TAOSADAPTERCORSALLOWWebSockets" --cors.exposeHeaders stringArray cors expose headers. Env "TAOSADAPTERExpose_Headers" --debug enable debug mode. Env "TAOSADAPTERDEBUG" (default true) --help Print this help message and exit --httpCodeServerError Use a non-200 http status code when server returns an error. Env "TAOSADAPTERHTTPCODESERVER_ERROR" --influxdb.enable enable influxdb. Env "TAOSADAPTERINFLUXDB_ENABLE" (default true) --log.enableRecordHttpSql whether to record http sql. Env "TAOSADAPTERLOGENABLERECORDHTTPSQL" --log.path string log path. Env "TAOSADAPTERLOGPATH" (default "/var/log/taos") --log.rotationCount uint log rotation count. Env "TAOSADAPTERLOGROTATION_COUNT" (default 30) --log.rotationSize string log rotation size(KB MB GB), must be a positive integer. Env "TAOSADAPTERLOGROTATIONSIZE" (default "1GB") --log.rotationTime duration log rotation time. Env "TAOSADAPTERLOGROTATIONTIME" (default 24h0m0s) --log.sqlRotationCount uint record sql log rotation count. Env "TAOSADAPTERLOGSQLROTATION_COUNT" (default 2) --log.sqlRotationSize string record sql log rotation size(KB MB GB), must be a positive

Env "TAOSADAPTERLOGSQLROTATION_SIZE" (default "1GB") --log.sqlRotationTime duration record sql log rotation time. Env "TAOSADAPTERLOGSQLROTATION_TIME" (default 24h0m0s) --logLevel string log level (panic fatal error warn warning info debug trace). Env "TAOSADAPTERLOG_LEVEL" (default "info") --monitor.collectDuration duration Set monitor duration. Env "TAOSADAPTERMONITORCOLLECTDURATION" (default 3s) --monitor.disable Whether to disable monitoring. Env "TAOSADAPTERMONITOR_DISABLE" --monitor.disableCollectClientIP Whether to disable collecting clientIP. Env "TAOSADAPTERMONITORDISABLECOLLECTCLIENTIP" --monitor.identity string The identity of the current instance, or 'hostname:port' if it is empty. Env "TAOSADAPTERMONITOR_IDENTITY" --monitor.incgroup Whether running in cgroup. Env "TAOSADAPTERMONITOR_INCGROUP" --monitor.password string TDengine password. Env "TAOSADAPTERMONITOR_PASSWORD" (default "taosdata") --monitor.pauseAllMemoryThreshold float Memory percentage threshold for pause all. Env "TAOSADAPTERMONITORPAUSEALLMEMORYTHRESHOLD" (default 80) --monitor.pauseQueryMemoryThreshold float Memory percentage threshold for pause query. Env "TAOSADAPTERMONITORPAUSEQUERYMEMORYTHRESHOLD" (default 70) --monitor.user string TDengine user. Env "TAOSADAPTERMONITORUSER" (default "root") --monitor.writeInterval duration Set write to TDengine interval. Env "TAOSADAPTERMONITORWRITE_INTERVAL" (default 30s) --monitor.writeToTD Whether write metrics to TDengine. Env "TAOSADAPTERMONITORWRITETO_TD" --nodeexporter.caCertFile string nodeexporter ca cert file path. Env "TAOSADAPTERNODEEXPORTERCACERTFILE" --nodeexporter.certFile string nodeexporter cert file path. Env "TAOSADAPTERNODEEXPORTERCERT_FILE" --nodeexporter.db string nodeexporter db name. Env "TAOSADAPTERNODEEXPORTERDB" (default "node_exporter") --nodeexporter.enable enable nodeexporter. Env "TAOSADAPTERNODEEXPORTERENABLE" --nodeexporter.gatherDuration duration nodeexporter gather duration. Env "TAOSADAPTERNODEEXPORTERGATHER_DURATION" (default 5s) --nodeexporter.httpBearerTokenString string nodeexporter http bearer token. Env "TAOSADAPTERNODEEXPORTERHTTPBEARERTOKEN_STRING" --nodeexporter.httpPassword string nodeexporter http password. Env "TAOSADAPTERNODEEXPORTERHTTP_PASSWORD" --nodeexporter.httpUsername string nodeexporter http username. Env "TAOSADAPTERNODEEXPORTERHTTP_USERNAME" --nodeexporter.insecureSkipVerify nodeexporter skip ssl check. Env "TAOSADAPTERNODEEXPORTERINSECURESKIPVERIFY" (default true) --nodeexporter.keyFile string nodeexporter cert key file path. Env "TAOSADAPTERNODEEXPORTERKEY_FILE" --nodeexporter.password string nodeexporter password. Env "TAOSADAPTERNODEEXPORTERPASSWORD" (default "taosdata") --nodeexporter.responseTimeout duration nodeexporter response timeout. Env "TAOSADAPTERNODEEXPORTERRESPONSE_TIMEOUT" (default 5s) --nodeexporter.ttl int nodeexporter data ttl. Env "TAOSADAPTERNODEEXPORTERTTL" --nodeexporter.urls strings nodeexporter urls. Env "TAOSADAPTERNODEEXPORTERURLS" (default [http://localhost:9100]) --nodeexporter.user string nodeexporter user. Env "TAOSADAPTERNODEEXPORTERUSER" (default "root") --opentsdb.enable enable opentsdb. Env "TAOSADAPTEROPENTSDB_ENABLE" (default true) --opentsdbtelnet.batchSize int opentsdbtelnet batch size. Env "TAOSADAPTEROPENTSDBTELNETBATCH_SIZE" (default 1) --opentsdbtelnet.dbs strings opentsdbtelnet db names. Env "TAOSADAPTEROPENTSDBTELNETDBS" (default [opentsdbtelnet,collectdtsdb,icinga2tsdb,tcollectortsdb]) --opentsdbtelnet.enable enable opentsdb telnet,warning: without auth info(default false). Env "TAOSADAPTEROPENTSDBTELNET_ENABLE" --opentsdbtelnet.flushInterval duration opentsdbtelnet flush interval (0s means not valid) . Env "TAOSADAPTEROPENTSDBTELNETFLUSH_INTERVAL" --opentsdbtelnet.maxTCPConnections int max tcp connections. Env "TAOSADAPTEROPENTSDBTELNETMAXTCP_CONNECTIONS" (default 250) --opentsdbtelnet.password string opentsdbtelnet password. Env "TAOSADAPTEROPENTSDBTELNETPASSWORD" (default "taosdata") --opentsdbtelnet.ports ints opentsdb telnet tcp port. Env "TAOSADAPTEROPENTSDBTELNET_PORTS" (default [6046,6047,6048,6049]) --opentsdbtelnet.tcpKeepAlive enable tcp keep alive. Env "TAOSADAPTEROPENTSDBTELNETTCPKEEP_ALIVE" --opentsdbtelnet.ttl int opentsdbtelnet data ttl. Env "TAOSADAPTEROPENTSDBTELNETTTL" --opentsdbtelnet.user string opentsdbtelnet user. Env "TAOSADAPTEROPENTSDBTELNETUSER" (default "root") --pool.idleTimeout duration Set idle connection timeout. Env "TAOSADAPTERPOOLIDLETIMEOUT" --pool.maxConnect int max connections to server. Env "TAOSADAPTERPOOLMAXCONNECT" --pool.maxIdle int max idle connections to server. Env "TAOSADAPTERPOOLMAXIDLE" -P, --port int http port. Env "TAOSADAPTERPORT" (default 6041) --prometheus.enable enable prometheus. Env "TAOSADAPTERPROMETHEUS_ENABLE" (default true) --restfulRowLimit int restful returns the maximum number of rows (-1 means no limit). Env "TAOSADAPTERRESTFULROWLIMIT" (default -1) --smlAutoCreateDB Whether to automatically create db when writing with schemaless. Env "TAOSADAPTERSMLAUTOCREATE_DB" --statsd.allowPendingMessages int statsd allow pending messages. Env "TAOSADAPTERSTATSDALLOWPENDING_MESSAGES" (default 50000) --statsd.db string statsd db name. Env "TAOSADAPTERSTATSDDB" (default "statsd") --statsd.deleteCounters statsd delete counter cache after gather. Env "TAOSADAPTERSTATSDDELETE_COUNTERS" (default true) --statsd.deleteGauges statsd delete gauge cache after gather. Env "TAOSADAPTERSTATSDDELETEGAUGES" (default true) --statsd.deleteSets statsd delete set cache after gather. Env "TAOSADAPTERSTATSDDELETESETS" (default true) --statsd.deleteTimings statsd delete timing cache after gather. Env "TAOSADAPTERSTATSDDELETETIMINGS" (default true) --statsd.enable enable statsd. Env "TAOSADAPTERSTATSD_ENABLE" (default true) --statsd.gatherInterval duration statsd gather interval. Env "TAOSADAPTERSTATSDGATHERINTERVAL" (default 5s) --statsd.maxTCPConnections int statsd max tcp connections. Env "TAOSADAPTERSTATSDMAXTCP_CONNECTIONS" (default 250) --statsd.password string statsd password. Env "TAOSADAPTERSTATSD_PASSWORD" (default "taosdata") --statsd.port int statsd server port. Env "TAOSADAPTERSTATSD_PORT" (default 6044) --statsd.protocol string statsd protocol [tcp or udp]. Env "TAOSADAPTERSTATSD_PROTOCOL" (default "udp") --statsd.tcpKeepAlive enable tcp keep alive. Env "TAOSADAPTERSTATSDTCPKEEPALIVE" --statsd.ttl int statsd data ttl. Env "TAOSADAPTERSTATSDTTL" --statsd.user string statsd user. Env "TAOSADAPTERSTATSD_USER" (default "root") --statsd.worker int statsd write worker. Env "TAOSADAPTERSTATSD_WORKER" (default 10) --taosConfigDir string load taos client config path. Env "TAOSADAPTERTAOSCONFIGFILE"

int When set to autocommit, the interval for message release is a multiple of the autocommit interval, with a default value of 2 and a minimum value of 1 and a maximum value of 10. Env "TAOSADAPTERTMQRELEASEINTERVALMULTIPLIERFOR_AUTOCOMMIT" (default 2) --version Print the version and exit ``` Note: Please set the following Cross-Origin Resource Sharing (CORS) parameters according to the actual situation when using a browser for interface calls. ```text AllowAllOrigins AllowOrigins AllowHeaders ExposeHeaders AllowCredentials AllowWebSockets ``` You do not need to care about these configurations if you do not make interface calls through the browser. For details on the CORS protocol, please refer to: or . See for sample configuration files. RESTful interface Compatible with InfluxDB v1 write interface Compatible with OpenTSDB JSON and telnet format writes - Seamless connection to collectd collectd is a system statistics collection daemon, please visit for more information. Seamless connection with StatsD StatsD is a simple yet powerful daemon for aggregating statistical information. Please visit for more information. Seamless connection with icinga2 icinga2 is a software that collects inspection result metrics and performance data. Please visit for more information. Seamless connection to TCollector TCollector is a client process that collects data from a local collector and pushes the data to OpenTSDB. Please visit for more information. Seamless connection to node_exporter node_export is an exporter for machine metrics. Please visit for more information. Support for Prometheus remoteread and remotewrite remoteread and remotewrite are interfaces for Prometheus data read and write from/to other data storage solution. Please visit for more information. Get table's VGroup ID. You can use any client that supports the http protocol to write data to or query data from TDengine by accessing the REST interface address `http://<fqdn>:6041/rest/sql`. See the for details. You can use any client that supports the http protocol to access the RESTful interface address `http://<fqdn>:6041/<APIEndPoint>` to write data in InfluxDB compatible format to TDengine. The EndPoint is as follows: ```text /influxdb/v1/write ``` Support InfluxDB query parameters as follows. `db` Specifies the database name used by TDengine `precision` The time precision used by TDengine `u` TDengine user name `p` TDengine password `ttl` The time to live of automatically created sub-table. This value cannot be updated. TDengine will use the ttl value of the first data of sub-table to create sub-table. For more information, please refer Note: InfluxDB token authorization is not supported at present. Only Basic authorization and query parameter validation are supported. Example: curl --request POST http://127.0.0.1:6041/influxdb/v1/write?db=test --user "root:taosdata" --data-binary "measurement,host=host1 field1=2i,field2=2.0 1577836800000000000" You can use any client that supports the http protocol to access the RESTful interface address `http://<fqdn>:6041/<APIEndPoint>` to write data in OpenTSDB compatible format to TDengine. The EndPoint is as follows: ```text /opentsdb/v1/put/json/<db> /opentsdb/v1/put/telnet/<db> ``` <CollectD /> <StatsD /> <Icinga2 /> <TCollector /> node_export is an exporter of hardware and OS metrics exposed by the \*NIX kernel used by Prometheus Enable the taosAdapter configuration `node_exporter.enable` Set the configuration of the node_exporter Restart taosAdapter <Prometheus /> You can call `http://<fqdn>:6041/rest/vgid?db=<db>&table=<table>` to get table's VGroup ID. taosAdapter will monitor its memory usage during operation and adjust it with two thresholds. Valid values are integers between 1 to 100, and represent a percentage of the system's physical memory. pauseQueryMemoryThreshold pauseAllMemoryThreshold Stops processing query requests when the `pauseQueryMemoryThreshold` threshold is exceeded. HTTP response content. code 503 body "query memory exceeds threshold" Stops processing all write and query requests when the `pauseAllMemoryThreshold` threshold is exceeded. HTTP response content. code 503 body "memory exceeds threshold" Resume the corresponding function when the memory falls back below the

Status check interface `http://<fqdn>:6041/-/ping` Normal returns `code 200` No parameter If memory exceeds pauseAllMemoryThreshold returns `code 503` Request parameter `action=query` returns `code 503` if memory exceeds `pauseQueryMemoryThreshold` or `pauseAllMemoryThreshold` Corresponding configuration parameter ```text monitor.collectDuration monitoring interval environment variable `TAOSMONITORCOLLECT_DURATION` (default value 3s) monitor.incgroup whether to run in cgroup (set to true for running in container) environment variable `TAOSMONITORINCGROUP` monitor.pauseAllMemoryThreshold memory threshold for no more inserts and queries environment variable `TAOSMONITORPAUSEALLMEMORY_THRESHOLD` (default 80) monitor.pauseQueryMemoryThreshold memory threshold for no more queries Environment variable `TAOSMONITORPAUSEQUERYMEMORY_THRESHOLD` (default 70) ``` You should adjust this parameter based on your specific application scenario and operation strategy. We recommend using monitoring software to monitor system memory status. The load balancer can also check the taosAdapter running status through this interface. taosAdapter collects HTTP-related metrics, CPU percentage, and memory percentage. Provides an interface conforming to . ```text http://<fqdn>:6041/metrics ``` taosAdapter supports writing the metrics of HTTP monitoring, CPU percentage, and memory percentage to TDengine. For configuration parameters | Configuration items | Description | Default values | |-|--|-| | monitor.collectDuration | CPU and memory collection interval | 3s | | monitor.identity | The current taosadapter identifier will be used if not set to `hostname:port` | | | monitor.incgroup | whether it is running in a cgroup (set to true for running in a container) | false | | monitor.writeToTD | Whether to write to TDengine | false | | monitor.user | TDengine connection username | root | | monitor.password | TDengine connection password | taosdata | | monitor.writeInterval | Write to TDengine interval | 30s | taosAdapter controls the number of results returned by the parameter `restfulRowLimit`, -1 means no limit, default is no limit. This parameter controls the number of results returned by the following interfaces: `http://<fqdn>:6041/rest/sql` `http://<fqdn>:6041/prometheus/v1/remote_read/:db` taosAdapter uses the parameter `httpCodeServerError` to set whether to return a non-200 http status code http status code other than when the C interface returns an error. When set to true, different http status codes will be returned according to the error code returned by C. For details, see HTTP Response Code chapter. Starting from version 3.0.4.0, the taosAdapter provides the parameter "smlAutoCreateDB" to control whether to automatically create DBs when writing with the schemaless protocol. The default value is false, which means that the DB will not be automatically created and the user needs to manually create the DB before performing schemaless writing. You can check the taosAdapter running status with the `systemctl status taosadapter` command. You can also adjust the level of the taosAdapter log output by setting the `--logLevel` parameter or the environment variable `TAOSADAPTERLOG_LEVEL`. Valid values are: panic, fatal, error, warn, warning, info, debug and trace. In TDengine server 2.2.x.x or earlier, the TDengine server process (taosd) contains an embedded HTTP service. As mentioned earlier, taosAdapter is a standalone software managed using `systemd` and has its own process ID. There are some configuration parameters and behaviors that are different between the two. See the following table for details. | # | embedded httpd | taosAdapter | comment | |-||-|| | 1 | httpEnableRecordSql | --logLevel=debug | | | 2 | httpMaxThreads | n/a | taosAdapter Automatically manages thread pools without this parameter | | 3 | telegrafUseFieldNum | See the taosAdapter telegraf configuration method | | | 4 | restfulRowLimit | restfulRowLimit | Embedded httpd outputs 10240 rows of data by default, the maximum allowed is 102400. taosAdapter also provides restfulRowLimit but it is not limited by default. You can configure it according to the actual scenario. | 5 | httpDebugFlag | Not applicable | httpdDebugFlag does not work for taosAdapter | | 6 | httpDBNameMandatory |