Jianghu Digital Platform
Login

Data. Data Caching

12003

1. Concept

  • Why Accelerate

In software development, acceleration typically refers to improving the performance of a system or application through various means. Performance issues may arise from slow data processing speeds, network transmission delays, complex computation processes, and other factors. By utilizing caching acceleration, some performance bottlenecks can be effectively resolved, enhancing the system's response speed and overall performance.

  • Advantages of Acceleration

    • Faster website access: Accelerating front-end and back-end resources can shorten the time users take to access pages, speed up page loading, improve user experience, and retain user traffic.

    • Reduced server load: By accelerating front-end and back-end resources, the server's burden and network bandwidth consumption can be reduced, preventing server crashes due to excessively large files.

    • Enhanced security: By accelerating front-end and back-end resources, some static resources can be placed on CDN acceleration nodes and encrypted during transmission using HTTPS, further improving access security.

    • Cost savings and improved scalability: By accelerating front-end and back-end resources, a CDN network can be used to speed up resource transmission. For large applications, the CDN network not only improves access speed but also transfers a significant amount of traffic to CDN nodes, alleviating server bandwidth pressure, while also saving server costs and enhancing application scalability.

2. Key Acceleration Points

  • Front-end Static Resources

    • Distribution via CDN: Store static resources (such as images, stylesheets, JavaScript files) on CDN nodes around the world, allowing users to access resources from closer nodes, thereby improving loading speed.

    • File compression and merging: Compress and merge front-end static resources to reduce file size, accelerating transmission and loading speed.

    • Using caching: Implement browser caching and server-side caching strategies to reduce duplicate requests and improve access speed.

  • Network Channel Acceleration

    • Using HTTPS: Adopt HTTPS protocol for secure transmission while utilizing the multiplexing feature of HTTP/2 protocol to reduce the number of connections and enhance concurrent access speed.

    • DNS resolution optimization: Configure DNS resolution reasonably, selectively using faster DNS servers to reduce domain name resolution time.

    • TCP connection optimization: Adjust TCP connection parameters, such as modifying TCP congestion control algorithms to optimize network transmission efficiency.

  • Back-end Data Caching Acceleration

    • Database caching: Use caching technology to store frequently queried data in the cache, reducing database access and improving data retrieval speed.

    • Result set caching: Implement caching strategies for some computationally expensive result sets to reduce computation time and enhance response speed.

    • Using caching middleware: Introduce caching middleware such as Redis or Memcached to provide high-speed, scalable caching services, accelerating data retrieval and storage.

These acceleration points cover multiple aspects of front-end resources, network channels, and back-end data processing, and their comprehensive application can significantly enhance overall system performance.

3. Static Resource Caching Acceleration

Front-end static resource caching acceleration can improve page loading speed, reduce server load, and bandwidth consumption. Below are several common methods.

  • Acceleration Methods

  1. Modify HTTP response headers: Add Expires and Cache-Control fields to the HTTP response headers to inform the browser and CDN caching servers about the expiration time or caching strategy for the resource. When the resource is within the cache validity period, the browser or CDN caching server will issue a 304 Not Modified response to retrieve the resource from the cache.

  2. Versioned file names: Use the file version number as the file name. If the file content changes, the file name changes, prompting the browser to re-fetch the resource and avoid using outdated resources. For example:

<link href="style.css?version=1.1" rel="stylesheet">

  1. Local caching: Use localStorage or sessionStorage to locally store static resources on the user's device, allowing faster access by reading directly from local cache on subsequent visits.

  2. Preloading: Use the rel attribute with the value preload in a link tag to pre-load static resources during page loading, enabling subsequent resource requests to complete more quickly.

  3. CDN acceleration: Utilize CDN caching servers to accelerate the transmission of static resources, improving page access speed and reducing bandwidth consumption.

  • JianghuJs Front-end Acceleration

There are two methods: through Egg framework configuration and the NPM package egg-static-cache.

  1. Through Egg framework configuration
  • Add configuration in config/config.default.js:
// config.default.js  
const path = require('path');  

static: {  
    dynamic: true,  
    preload: false,  
    maxAge: 31536000,  
    buffer: true,  
    dir: [  
        { prefix: `/${appId}/public/`, dir: path.join(appInfo.baseDir, 'app/public') },  
        { prefix: `/${appId}/public/`, dir: path.join(eggJianghuDir, 'app/public') },  
    ],  
},

Here, maxAge indicates the caching duration for static resources (in seconds), prefix indicates the prefix, dir indicates the directory for static resources, dynamic indicates whether caching is enabled, preload indicates whether to preload the cache at startup, maxFiles indicates the maximum number of cached files, and buffer indicates whether to cache in memory. 

// plugin.js  
exports.static = true;
  • enable indicates whether caching is enabled.
  1. NPM package egg-static-cache
  • Install the egg-static-cache dependency:
npm install egg-static-cache --save
  • Add configuration in config.default.js:
// config.default.js  
exports.staticCache = {  
  maxAge: 31536000,  
  dynamic: true,  
  prefix: '/public/',  
  gzip: true,  
};
  • Add configuration to enable egg-static-cache in plugin.js:
// plugin.js  
exports.static = true;  
exports.staticCache = {  
  enable: true,  
  package: 'egg-static-cache',  
};

  • enable indicates whether caching is enabled, and package indicates the name of the dependent NPM package.

  • Specify access to static resources in Router:

// Router.js  
app.router.get('/public/(*)', app.controller.home.resources);
  • Implement front-end resource caching in Controller:
// Controller.js  
const staticCache = require('egg-static-cache');  
async resources() {  
    const { app, ctx } = this;  
    await staticCache(app, ctx, {  
         prefix: '/public/'  
    })  
}

4. Data Caching Acceleration

  • JianghuJs Back-end Acceleration (_cache)

The JianghuJs framework minimizes weight by avoiding the introduction of middleware like Redis, thus using a cache table to address data caching issues, which users can expand upon as needed.

  • Usage:

Users can use scheduled tasks or other methods to process data and store it in the _cache table, which can then be accessed directly.

  • Table Design
CREATE TABLE `_cache` (  
  `id` int(11) NOT NULL AUTO_INCREMENT,  
  `userId` varchar(255) NOT NULL COMMENT 'User ID',  
  `content` longtext COMMENT 'Cached data',  
  `recordStatus` varchar(255) DEFAULT 'active',  
  `operation` varchar(255) DEFAULT 'insert' COMMENT 'Operation; insert, update, jhInsert, jhUpdate, jhDelete, jhRestore',  
  `operationByUserId` varchar(255) DEFAULT NULL COMMENT 'Operator user ID',  
  `operationByUser` varchar(255) DEFAULT NULL COMMENT 'Operator username',  
  `operationAt` varchar(255) DEFAULT NULL COMMENT 'Operation time; E.g: 2021-05-28T10:24:54+08:00 ',  
  PRIMARY KEY (`id`)  
) ENGINE = InnoDB COMMENT = 'Cache table';