fulltext

Searches data using a full-text index.

Command properties

Syntax

BOOLEAN_EXPR

Boolean expression to search for. Enclose each search term in double quotes and combine with and, or, and not. Search strings are case-insensitive.

• Simple search: "192.0.2.1"
• AND: "192.0.2.1" and "80"
• OR: "error" or "failed"
• NOT: "error" not "debug"
• Functions: range, iprange (see the Search functions section under Description)

You can use a subquery enclosed in square brackets ([ ]) inside the Boolean expression. The subquery runs first, before the index search, and all terms appearing in its results are searched. Because index search slows down as the subquery returns more terms, use the fields command inside the subquery to retrieve only the fields you need.

Options

from=DATE

Optional. Start time of the search range. Enter in yyyyMMddHHmmss format; omitted time parts are automatically filled with zeros.

to=DATE

Optional. End time of the search range. Enter in yyyyMMddHHmmss format; omitted time parts are automatically filled with zeros.

duration=INT{s|m|h|d|mon}

Optional. Limits the search to a specified time window from the current time. For example, 10m limits the range to the last 10 minutes.

offset=INT

Optional. Number of search results to skip (default: 0).

limit=INT

Optional. Maximum number of search results to return. If not specified, all results are returned.

order={asc|desc}

Sort order of search results (default: desc).

• desc: Search from newest data first
• asc: Search from oldest data first

tt={true|false}

Optional. Automatically splits the search string using each index's tokenizer before searching. For example, fulltext tt=t "a.b.c.d" is equivalent to fulltext "a" and "b" and "c" and "d".

raw={true|false}

Optional. If set to true, retrieves raw log data. Requires cluster administrator permission. (default: false)

Target

QUALIFIER

Specifies the table or index to search. The format is [NODE:][TABLE[.FIELD]] or * (all tables).

• * — Search all indexes of all tables
• table_name — Search all indexes of the specified table
• table_name.field_name — Search the index of a specific field in the specified table
• data_node:table_name — Search a table on a specific Data Node

Input fields

None (as a driver query, this command can only be used as the first command in a pipeline)

Output fields

All fields stored in the table are output. System fields are:

Error codes

Parse errors

Runtime errors

N/A

Description

The fulltext command uses a full-text index to search data stored in tables. Because it utilizes indexes, it can search large datasets faster than combining the table and search commands. In the 4.0 series starting with 4.0.2609.0, if a partition's index has been consolidated into a separately merged index, the merged index is included in the search scope the same as the original partitions, so search results are the same regardless of whether merging has occurred. The 5.0 series (through 5.0.2605.2) does not have this index-merging feature.

In a distributed environment, index searches run on each Data Node. The fulltext command provides faster search performance than the table command.

Notes:

• Specifying the same table or index multiple times causes duplicate results
• Search speed may decrease when subquery results are large

Matched record IDs are read sequentially from the index and the actual log records are fetched asynchronously. Log fetching is processed through a single-threaded queue with a capacity of 10. If the queue remains full for an extended period, a timeout error may occur.

Search functions

The fulltext command provides index-only functions usable inside BOOLEAN_EXPR. These functions only work on indexed fields and cannot be called from eval or other commands.

The range function cannot have a difference of more than 65,535 between MAX and MIN. The iprange function supports both IPv4 and IPv6, and both arguments must use the same format.

Examples

1. Search logs containing a specific attacker IP

   fulltext "203.0.113.42" from WEB_APACHE
   

   Searches the WEB_APACHE table for all access records containing the attacker IP address 203.0.113.42.

2. Search for malicious activity patterns using AND

   fulltext "203.0.113.42" and "sql injection" from WEB_APACHE
   

   Searches the WEB_APACHE table for suspicious requests containing both the attacker IP 203.0.113.42 and the keyword sql injection.

3. Search error events with a time range and record limit

   fulltext duration=1h limit=1000 "connection timeout" or "service unavailable" from SYSTEM_EVENT_LOG
   

   Searches the SYSTEM_EVENT_LOG table for system error events containing connection timeout or service unavailable within the past hour, returning up to 1,000 results.

Change history