Jianghu Digital Platform
Login

Application Logs

12003

The importance of logs in web development is undeniable; they play a crucial role in monitoring the application's operational status and troubleshooting issues.

The framework comes with powerful enterprise-level log support, provided by the egg-logger module.

Key features include:

  • Log levels.
  • Unified error logs: All ERROR level logs printed using .error() in any logger will be directed to a unified error log file for easier tracking.
  • Separation of startup logs and runtime logs.
  • Custom logs.
  • Multi-process logging.
  • Automatic log rotation.
  • High performance.

Log Path

  • All log files are by default placed in the ${appInfo.root}/logs/${appInfo.name} path, for example, /home/admin/logs/example-app.
  • In local development environments (env: local) and unit testing environments (env: unittest), to avoid conflicts and for centralized management, logs will be printed in the logs directory under the project directory, for example, /path/to/example-app/logs/example-app.

If you want to customize the log path:

// config/config.${env}.js
exports.logger = {
  dir: '/path/to/your/custom/log/dir',
};

Log Categories

The framework includes several types of logs, each used in different scenarios:

  • appLogger ${appInfo.name}-web.log, for example, example-app-web.log, application-related logs intended for application developers. We use it in most cases.
  • coreLogger egg-web.log: Framework core and plugin logs.
  • errorLogger common-error.log: Generally, this is not used directly; any logs output by .error() from any logger will be redirected here, focusing on locating exceptions by reviewing this log.
  • agentLogger egg-agent.log: Logs from the agent process, where the framework and plugins that utilize the agent process to execute tasks will print some logs here.

If you want to customize the names of the above log files, you can override the default values in the config file:

// config/config.${env}.js
module.exports = appInfo => {
  return {
    logger: {
      appLogName: `${appInfo.name}-web.log`,
      coreLogName: 'egg-web.log',
      agentLogName: 'egg-agent.log',
      errorLogName: 'common-error.log',
    },
  };
};

How to Print Logs

Context Logger

If we need to print logs while processing requests, we use the Context Logger to record logs related to web behavior.

Each log line will automatically record some basic information about the current request, such as [$userId/$ip/$traceId/${cost}ms $method $url].

ctx.logger.debug('debug info');
ctx.logger.info('some request data: %j', ctx.request.body);
ctx.logger.warn('Warning!');

// Error log recording will directly log the complete stack trace of the error and output it to errorLog
// To ensure exceptions are traceable, all thrown exceptions must be of type Error, as only Error types carry stack information to pinpoint the issue.
ctx.logger.error(new Error('whoops'));

For framework and plugin developers, ctx.coreLogger can also be used.

For example:

ctx.coreLogger.info('info');

App Logger

If we want to perform some application-level logging, such as recording some data during the startup phase, we can use the App Logger.

// app.js
module.exports = app => {
  app.logger.debug('debug info');
  app.logger.info('Startup took %d ms', Date.now() - start);
  app.logger.warn('Warning!');

  app.logger.error(someErrorObj);
};

For framework and plugin developers, app.coreLogger can also be used.

// app.js
module.exports = app => {
  app.coreLogger.info('Startup took %d ms', Date.now() - start);
};

Agent Logger

When developing frameworks and plugins, there may be a need to run code in the Agent process, in which case we use agent.coreLogger.

// agent.js
module.exports = agent => {
  agent.logger.debug('debug info');
  agent.logger.info('Startup took %d ms', Date.now() - start);
  agent.logger.warn('Warning!');

  agent.logger.error(someErrorObj);
};

For more details about the Agent process, please refer to Multi-Process Model.

Log File Encoding

The default encoding is utf-8, which can be overridden as follows:

// config/config.${env}.js
exports.logger = {
  encoding: 'gbk',
};

Log File Format

Set the output format to JSON for easier analysis by log monitoring systems.

// config/config.${env}.js
exports.logger = {
  outputJSON: true,
};

Log Levels

Logs are categorized into five levels: NONE, DEBUG, INFO, WARN, and ERROR.

Logs printed to files will also be printed to the terminal for development convenience.

File Log Levels

By default, only INFO and above (i.e., WARN and ERROR) logs will be output to files.

You can configure the log levels output to files as follows:

  • Print all log levels to files:
// config/config.${env}.js
exports.logger = {
  level: 'DEBUG',
};
  • Disable all logs output to files:
// config/config.${env}.js
exports.logger = {
  level: 'NONE',
};

Printing Debug Logs in Production

To avoid performance issues caused by printing debug logs from some plugins in production, the default setting prohibits printing DEBUG level logs in production. If there is a need to print DEBUG logs for debugging in production, the allowDebugAtProd configuration option must be enabled.

// config/config.prod.js
exports.logger = {
  level: 'DEBUG',
  allowDebugAtProd: true,
};

Terminal Log Levels

By default, only INFO and above (i.e., WARN and ERROR) logs will be output to the terminal. These logs are only printed to the terminal in local and unittest environments.

You can configure the log levels output to the terminal as follows:

  • Print all log levels to the terminal:
// config/config.${env}.js
exports.logger = {
  consoleLevel: 'DEBUG',
};
  • Disable all logs output to the terminal:
// config/config.${env}.js
exports.logger = {
  consoleLevel: 'NONE',
};
  • For performance reasons, terminal log output is disabled by default in production environments. If needed, you can enable it with the following configuration (not recommended):
// config/config.${env}.js
exports.logger = {
  disableConsoleAfterReady: false,
};

Custom Logs

Adding Custom Logs

Generally, applications do not need to configure custom logs, as excessive or scattered logging can lead to distraction and make it harder to manage and troubleshoot issues.

If there is indeed such a need, you can refer to the following configuration:

// config/config.${env}.js
const path = require('path');

module.exports = appInfo => {
  return {
    customLogger: {
      xxLogger: {
        file: path.join(appInfo.root, 'logs/xx.log')
      }
    }
  };
};

You can obtain the custom log object using app.getLogger('xxLogger') or ctx.getLogger('xxLogger'). The final printed log format is similar to that of coreLogger.

Custom Log Format

// config/config.${env}.js
const path = require('path');

module.exports = appInfo => {
  return {
    customLogger: {
      xxLogger: {
        file: path.join(appInfo.root, 'logs/xx.log'),
        formatter(meta) {
          return `[${meta.date}] ${meta.message}`;
        },
        contextFormatter(meta) {
          return `[${meta.date}] [${meta.ctx.method} ${meta.ctx.url}] ${meta.message}`;
        }
      }
    }
  };
};

Advanced Custom Logs

Logs are by default printed to log files and also printed to the terminal during local development. However, sometimes we need to print logs to other media, such as reporting error logs to common-error.log while also sending them to a third-party service.

First, we define a log transport channel that represents the third-party log service.

const util = require('util');
const Transport = require('egg-logger').Transport;

class RemoteErrorTransport extends Transport {
  // Define the log method. In this method, the log is reported to the remote service.
  log(level, args) {
    let log;
    if (args[0] instanceof Error) {
      const err = args[0];
      log = util.format(
        '%s: %s\n%s\npid: %s\n',
        err.name,
        err.message,
        err.stack,
        process.pid
      );
    } else {
      log = util.format(...args);
    }

    this.options.app.curl('http://url/to/remote/error/log/service/logs', {
      data: log,
      method: 'POST'
    })
    .catch(console.error);
  }
}

// In app.js, add transport to errorLogger, so each log will also be printed to this transport.
app.getLogger('errorLogger').set('remote', new RemoteErrorTransport({ level: 'ERROR', app }));

In the above code example, while it is relatively simple, performance considerations must be taken into account in actual use. Typically, a strategy of temporarily storing logs in memory and uploading them periodically is employed to optimize performance.

Log Rotation

One of the most common requirements for enterprise-level logging is automatic log rotation for easier management. The framework supports log rotation through the egg-logrotator plugin.

Daily Rotation

This is the default log rotation method of the framework, which rotates logs daily at 00:00 according to the .log.YYYY-MM-DD filename format.

For example, for appLog, when the current log being written is example-app-web.log, at 00:00, the logs from the past day will be rotated into separate files in the format example-app-web.log.YYYY-MM-DD.

Size-Based Rotation

We can also choose to rotate logs based on file size. For example, when a file exceeds 2GB, it will be rotated.

For instance, if we want to rotate egg-web.log based on size:

// config/config.${env}.js
const path = require('path');

module.exports = (appInfo) => {
  return {
    logrotator: {
      filesRotateBySize: [
        path.join(appInfo.root, 'logs', appInfo.name, 'egg-web.log'),
      ],
      maxFileSize: 2 * 1024 * 1024 * 1024,
    },
  };
};

Logs added to filesRotateBySize will no longer be rotated daily.

Hourly Rotation

We can also choose to rotate logs hourly, which is similar to the default daily rotation but occurs every hour.

For example, if we want to rotate common-error.log hourly:

// config/config.${env}.js
const path = require('path');

module.exports = (appInfo) => {
  return {
    logrotator: {
      filesRotateByHour: [
        path.join(appInfo.root, 'logs', appInfo.name, 'common-error.log'),
      ],
    },
  };
};

Logs added to filesRotateByHour will also no longer be rotated daily.

Performance

Typically, web access is high-frequency, and writing logs directly to disk for each output can lead to frequent disk I/O operations. To improve performance, our file logging strategy is:

Logs are synchronously written to memory and asynchronously flushed to disk every period (default 1 second).

For more details, please refer to egg-logger and egg-logrotator.