Storage

Overview

Storage is a part of the data lifecycle. For an overview of the data lifecycle, refer to here.

In Settings > Cluster > Storage, you can connect to storage services such as AWS S3 and KakaoCloud Object Storage for the Cold tier, set rollover schedules and bandwidth limit, and configure maximum capacity for each storage tier.

Object Storage Profile

The object storage profile is a dedicated configuration used to connect to object storage services provided by AWS and KakaoCloud. You can set up the object storage profile either during Logpresso Sonar installation or later via the web console.

Object storage profile

Caution
Once configured, the object storage settings cannot be changed. Please pre-configure the object storage environment that Logpresso Sonar will connect to and verify the necessary information.
Web Installer

During the Logpresso Sonar installation, you can configure AWS S3 or Kakao Object Storage in the object storage setup step.

  • To skip the object storage setup, click Skip.
  • To verify if the object storage settings are correct, click Test.
Web Console

To set up object storage profile in the web console:

  1. Click Settings in Settings > Cluster > Storage.

    Web Console - Object Storage Setup

  2. Enter the required information for the object storage connection in the Object Storage Profile and click Test.

    Web Console - Object Storage Profile Information Input

    • Type: Storage service provider (choose between AWS and Kakao; default: AWS)
      • AWS: AWS S3
      • Kakao: KakaoCloud Object Storage
    • Access Key: Access key for connecting to the storage service
    • Secret Key: Secret access key for connecting to the storage service
    • Endpoint: Access address for the storage service
    • Bucket: Name of the storage bucket
    • User ID: (when Type is Kakao) User unique ID that can be found in KakaoCloud Console > User Profile > Account Information
    • Project ID: (when Type is Kakao) ID of the KakaoCloud project

  3. If the test result indicates a successful connection to the storage, click Confirm.

    Web Console - Object Storage Connection Test Success

    If the connection to the storage is unsuccessful, check the cause and take corrective action before clicking Test again.

    Web Console - Object Storage Connection Test Failure

    • Verify the network connection between the Logpresso Sonar node and the storage service.
    • Ensure that the object storage being used is configured correctly.
    • Confirm that the object storage connection information, such as the access key and secret access key, is entered correctly.
Common Settings (Rollover Settings)

In the Rollover Settings, you can configure the rollover execution schedule and the bandwidth limit to be applied during the rollover. After making changes, click Save. If you do not click Save, the changes will be lost.

Rollover Settings

  • Rollover Schedule: The schedule for executing the rollover. Click "…" and specify the execution time according to CRON syntax in the Task Cronizer (default: unspecified).
  • Rollover Throttling: The file transfer speed to be applied during the rollover (choose from No Throttling, Max 320MB/s, Max 100MB/s, Max 40MB/s, Max 16MB/s; default: Max 40MB/s).
Note
It is recommended to schedule the execution during late-night hours when business activities are minimal. Initially, increase the bandwidth limit after introducing the data lifecycle. Once the data is appropriately distributed across the storage tiers, you can reduce the bandwidth limit back to the default level.
Maximum Storage Capacity

The maximum storage capacity is specified for each storage tier. For Hot and Warm tier storage, it is based on the local storage capacity of each node, while for Cold tier storage, it is calculated based on the data volume stored at the specified URI. After making changes, click Save. If you do not click Save, the changes will be lost.

Maximum Storage Capacity

  • Specify the capacity for Hot, Warm, and Cold tiers and click Save.
  • The maximum capacity can be specified in MB, GB, or TB, or you can select Unlimited.
  • When the maximum capacity is set to Unlimited, data will not be deleted even if it exceeds the maximum storage capacity.
  • If the storage device mounted in the Warm or Cold tier is WORM or cloud storage, it can be verified below the maximum capacity.

This setting only displays tiers that have a defined lifecycle. If no lifecycle is defined, the maximum storage capacity settings will not be visible.