`
sundoctor
  • 浏览: 323778 次
  • 性别: Icon_minigender_1
  • 来自: 北京
社区版块
存档分类
最新评论

扩展spring boot cache实现redis一二级分布式缓存

 
阅读更多

 

系统为了提高数据访问速度,先将数据加载到redis缓存中,但是每次从缓存获取数据,要通过网络访问才能获取,效率还是不够逆天快。如果访问量很大,并发很高,性能不够快不说,还容易造成reids负载过高,redis的主机出现各种物理故障。因此,可以在redis前增加本地一级缓存,本地一级缓存和系统应用在同一个JVM内,这样速度最快,redis退居二线当作二级缓存。每次请求先从一级缓存读取数据,一级缓存没有数据,再从二级缓存读取,并同步到一级缓存里,通过redis的消息发布订阅通知其他client机器更新缓存。这同CPU的一级缓存,二级缓存是一个道理。

 

本文并不涉及spring boot cache的详细使用介绍,需要熟悉spring boot cache基本使用。对于spring boot cache详细使用介绍请上度娘。扩展比较简单,闲话少说,直接上代码。

 

pom.xml,加入spring boot cache依赖,本地一级缓存使用ehcache3,二级缓存使用redis

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>2.0.0.M7</version>
    </parent>
    <groupId>springboot</groupId>
    <artifactId>springboot-2level-cache</artifactId>
    <version>1.0-SNAPSHOT</version>

    <dependencies>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-cache</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-data-redis</artifactId>
        </dependency>
        <dependency>
            <groupId>redis.clients</groupId>
            <artifactId>jedis</artifactId>
        </dependency>
        <dependency>
            <groupId>javax.cache</groupId>
            <artifactId>cache-api</artifactId>
        </dependency>
        <dependency>
            <groupId>org.ehcache</groupId>
            <artifactId>ehcache</artifactId>
        </dependency>

        <!-- Only used to expose cache metrics -->
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-actuator</artifactId>
        </dependency>

        <!-- Test -->
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
    </dependencies>
    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
            </plugin>
        </plugins>
    </build>
    <repositories>
        <repository>
            <id>spring-milestones</id>
            <name>Spring Milestones</name>
            <url>https://repo.spring.io/libs-milestone</url>
            <snapshots>
                <enabled>false</enabled>
            </snapshots>
        </repository>
    </repositories>

</project>

 

appliaction.properties里配置属性

 

spring.cache.type=redis
spring.ext.cache.name=countries
spring.ext.cache.redis.topic=cache

 

本地一级缓存使用ehcache3ehcache3.xml配置

 

<config xmlns='http://www.ehcache.org/v3'
		xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
		xmlns:jsr107="http://www.ehcache.org/v3/jsr107"
		xsi:schemaLocation="http://www.ehcache.org/v3 http://www.ehcache.org/schema/ehcache-core-3.0.xsd
							http://www.ehcache.org/v3/jsr107 http://www.ehcache.org/schema/ehcache-107-ext-3.0.xsd">
	<cache alias="countries">
		<expiry>
			<ttl unit="seconds">600</ttl>
		</expiry>
		<heap unit="entries">200</heap>
		<jsr107:mbeans enable-statistics="true"/>
	</cache>

</config>

 

创建LocalRedisCache 继承RedisCache,这里是实现redis一二级分布式缓存的核心,重载RedisCachegetputevictclean等方法,增加本地一级缓存的读写,增加pub方法,通过redis发布缓存更新消息通知其他 redis clent 更新缓存,增加sub方法,获取缓存更新消息更新缓存。

 

package org.springframework.data.redis.cache;

import org.springframework.cache.Cache;
import org.springframework.data.redis.core.RedisOperations;
import org.springframework.util.Assert;

import java.util.concurrent.Callable;

/**
 * extends RedisCache,增加本地一级缓存,redis作为二级缓存
 */
public class LocalRedisCache extends RedisCache {

    private final Cache localCache;//本地一级缓存
    private final RedisOperations redisOperations;//配合topicName,发布缓存更新消息
    private final String topicName;//redis topic ,发布缓存更新消息通知其他client更新缓存

    /**
     * Create new {@link RedisCache}.
     *
     * @param name            must not be {@literal null}.
     * @param cacheWriter     must not be {@literal null}.
     * @param cacheConfig     must not be {@literal null}.
     * @param localCache      must not be {@literal null}.
     * @param redisOperations must not be {@literal null}.
     * @param topicName       must not be {@literal null}.
     */
    protected LocalRedisCache(String name, RedisCacheWriter cacheWriter, RedisCacheConfiguration cacheConfig,
                              Cache localCache, RedisOperations redisOperations, String topicName) {

        super(name, cacheWriter, cacheConfig);

        Assert.notNull(localCache, "localCache must not be null!");
        Assert.notNull(redisOperations, "redisOperations must not be null!");
        Assert.hasText(topicName, "topicName must not be empty!");

        this.localCache = localCache;
        this.redisOperations = redisOperations;
        this.topicName = topicName;
    }

    @Override
    public synchronized <T> T get(Object key, Callable<T> valueLoader) {
        //先读取本地一级缓存
        T value = localCache.get(key, valueLoader);
        if (value == null) {
            //本地一级缓存不存在,读取redis二级缓存
            value = super.get(key, valueLoader);
            if (value != null) {
                //redis二级缓存存在,存入本地一级缓存
                localCache.put(key, value);
                //发布缓存更新消息通知其他client更新缓存
                pub(new UpdateMessage(key, value, UpdateMessage.Type.PUT));
            }
        }
        return value;
    }

    @Override
    public void put(Object key, Object value) {
        super.put(key, value);
        localCache.put(key, value);
        pub(new UpdateMessage(key, value, UpdateMessage.Type.PUT));
    }

    @Override
    public ValueWrapper putIfAbsent(Object key, Object value) {
        ValueWrapper vw1 = localCache.putIfAbsent(key, value);
        ValueWrapper vw2 = super.putIfAbsent(key, value);

        pub(new UpdateMessage(key, value, UpdateMessage.Type.PUTIFABSENT));

        return vw1 == null ? vw2 : vw1;
    }

    @Override
    public void evict(Object key) {
        localCache.evict(key);
        super.evict(key);
        pub(new UpdateMessage(key, UpdateMessage.Type.REMOVE));
    }

    @Override
    public void clear() {
        localCache.clear();
        super.clear();
        pub(new UpdateMessage(UpdateMessage.Type.CLEAN));
    }

    @Override
    public ValueWrapper get(Object key) {
        ValueWrapper valueWrapper = localCache.get(key);
        if (valueWrapper == null) {
            valueWrapper = super.get(key);
            if (valueWrapper != null) {
                localCache.put(key, valueWrapper.get());
                pub(new UpdateMessage(key, valueWrapper.get(), UpdateMessage.Type.PUT));
            }
        }
        return valueWrapper;
    }

    @Override
    public <T> T get(Object key, Class<T> type) {
        T value = localCache.get(key, type);
        if (value == null) {
            value = super.get(key, type);
            if (value != null) {
                localCache.put(key, value);
                pub(new UpdateMessage(key, value, UpdateMessage.Type.PUT));
            }
        }
        return value;
    }

    /**
     * 更新缓存
     *
     * @param updateMessage
     */
    public void sub(final UpdateMessage updateMessage) {

        if (updateMessage.getType() == UpdateMessage.Type.CLEAN) {
            //清除所有缓存
            localCache.clear();
            super.clear();
        } else if (updateMessage.getType() == UpdateMessage.Type.PUT) {
            //更新缓存
            localCache.put(updateMessage.getKey(), updateMessage.getValue());
            super.put(updateMessage.getKey(), updateMessage.getValue());
        } else if (updateMessage.getType() == UpdateMessage.Type.PUTIFABSENT) {
            //更新缓存
            localCache.putIfAbsent(updateMessage.getKey(), updateMessage.getValue());
            super.putIfAbsent(updateMessage.getKey(), updateMessage.getValue());
        } else if (updateMessage.getType() == UpdateMessage.Type.REMOVE) {
            //删除缓存
            localCache.evict(updateMessage.getKey());
            super.evict(updateMessage.getKey());
        }
    }

    /**
     * 通知其他 redis clent 更新缓存
     *
     * @param message
     */
    private void pub(final UpdateMessage message) {
        this.redisOperations.convertAndSend(topicName, message);
    }

}

 

创建新的缓存管理器,命名为LocalRedisCacheManager继承了Spring BootRedisCacheManager,重载createRedisCache方法。

 

package org.springframework.data.redis.cache;

import org.springframework.cache.Cache;
import org.springframework.data.redis.connection.RedisConnectionFactory;
import org.springframework.data.redis.core.RedisOperations;
import org.springframework.util.Assert;

import java.util.Map;

public class LocalRedisCacheManager extends RedisCacheManager {

    private final RedisConnectionFactory connectionFactory;
    private final Cache localCache;
    private final RedisOperations redisOperations;
    private final String topicName;

    public LocalRedisCacheManager(RedisCacheWriter cacheWriter, RedisCacheConfiguration defaultCacheConfiguration,
                                  RedisConnectionFactory connectionFactory, Cache localCache, RedisOperations redisOperations,
                                  String topicName) {
        super(cacheWriter, defaultCacheConfiguration);


        this.connectionFactory = connectionFactory;
        this.localCache = localCache;
        this.redisOperations = redisOperations;
        this.topicName = topicName;

        check();
    }

    public LocalRedisCacheManager(RedisCacheWriter cacheWriter, RedisCacheConfiguration defaultCacheConfiguration,
                                  RedisConnectionFactory connectionFactory, Cache localCache,
                                  RedisOperations redisOperations, String topicName, String... initialCacheNames) {
        super(cacheWriter, defaultCacheConfiguration, initialCacheNames);


        this.connectionFactory = connectionFactory;
        this.localCache = localCache;
        this.redisOperations = redisOperations;
        this.topicName = topicName;

        check();
    }

    public LocalRedisCacheManager(RedisCacheWriter cacheWriter, RedisCacheConfiguration defaultCacheConfiguration,
                                  Map<String, RedisCacheConfiguration> initialCacheConfigurations,
                                  RedisConnectionFactory connectionFactory, Cache localCache,
                                  RedisOperations redisOperations, String topicName) {
        super(cacheWriter, defaultCacheConfiguration, initialCacheConfigurations);

        this.connectionFactory = connectionFactory;
        this.localCache = localCache;
        this.redisOperations = redisOperations;
        this.topicName = topicName;

        check();
    }

    @Override
    protected RedisCache createRedisCache(String name, RedisCacheConfiguration cacheConfig) {
        return new LocalRedisCache(name, new DefaultRedisCacheWriter(connectionFactory),
                cacheConfig != null ? cacheConfig : RedisCacheConfiguration.defaultCacheConfig(),
                localCache, redisOperations, topicName);
    }

    public static LocalRedisCacheManager create(RedisConnectionFactory connectionFactory, Cache localCache, RedisOperations redisOperations, String topicName) {

        Assert.notNull(localCache, "localCache must not be null");
        Assert.notNull(connectionFactory, "connectionFactory must not be null");
        Assert.notNull(redisOperations, "redisOperations must not be null");
        Assert.notNull(topicName, "topicName must not be null");

        return new LocalRedisCacheManager(new DefaultRedisCacheWriter(connectionFactory),
                RedisCacheConfiguration.defaultCacheConfig(), connectionFactory, localCache, redisOperations, topicName);
    }


    private void check() {
        Assert.notNull(localCache, "localCache must not be null");
        Assert.notNull(connectionFactory, "connectionFactory must not be null");
        Assert.notNull(redisOperations, "redisOperations must not be null");
        Assert.notNull(topicName, "topicName must not be null");
    }


}

 

 

缓存配置实现,需要使用注解 @EnableCaching 打开缓存功能

package org.springframework.data.redis.cache;

import org.springframework.beans.factory.annotation.Value;
import org.springframework.cache.CacheManager;
import org.springframework.cache.annotation.EnableCaching;
import org.springframework.cache.jcache.JCacheCacheManager;
import org.springframework.cache.jcache.JCacheManagerFactoryBean;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Primary;
import org.springframework.core.io.ClassPathResource;
import org.springframework.core.io.Resource;
import org.springframework.data.redis.connection.Message;
import org.springframework.data.redis.connection.MessageListener;
import org.springframework.data.redis.connection.jedis.JedisConnectionFactory;
import org.springframework.data.redis.core.RedisTemplate;
import org.springframework.data.redis.listener.PatternTopic;
import org.springframework.data.redis.listener.RedisMessageListenerContainer;
import org.springframework.data.redis.listener.adapter.MessageListenerAdapter;

import java.io.IOException;

@Configuration
@EnableCaching
public class CacheConfig {

    @Value("${spring.ext.cache.name:countries}")
    private String localCacheName;

    @Value("${spring.ext.cache.redis.topic:cache}")
    private String topicName;

    @Bean
    public CacheManager jCacheCacheManager() {
        return new JCacheCacheManager(jCacheManagerFactoryBean().getObject());
    }

    @Bean
    public JCacheManagerFactoryBean jCacheManagerFactoryBean() {
        JCacheManagerFactoryBean jCacheManagerFactoryBean = new JCacheManagerFactoryBean();
        Resource resource = new ClassPathResource("ehcache3.xml");
        try {
            jCacheManagerFactoryBean.setCacheManagerUri(resource.getURI());
        } catch (IOException e) {
            throw new RuntimeException(e);
        }
        return jCacheManagerFactoryBean;
    }

    @Bean
    public JedisConnectionFactory jedisConnectionFactory() {
        JedisConnectionFactory factory = new JedisConnectionFactory();
        return factory;
    }

    @Bean
    public RedisTemplate redisTemplate(JedisConnectionFactory jedisConnectionFactory) {
        RedisTemplate redisTemplate = new RedisTemplate();
        redisTemplate.setConnectionFactory(jedisConnectionFactory);
        return redisTemplate;
    }

    @Primary
    @Bean
    public RedisCacheManager redisCacheManager(JedisConnectionFactory jedisConnectionFactory,
                                               RedisTemplate redisTemplate) {
        RedisCacheManager cacheManager = LocalRedisCacheManager.create(jedisConnectionFactory,
                jCacheCacheManager().getCache(localCacheName), redisTemplate, topicName);
        return cacheManager;
    }


    @Bean
    public RedisMessageListenerContainer container(JedisConnectionFactory jedisConnectionFactory,
                                                   MessageListenerAdapter listenerAdapter) {

        RedisMessageListenerContainer container = new RedisMessageListenerContainer();
        container.setConnectionFactory(jedisConnectionFactory);
        container.addMessageListener(listenerAdapter, new PatternTopic(topicName));

        return container;
    }

    @Bean
    public MessageListenerAdapter listenerAdapter(LocalRedisCacheManager cacheManager, RedisTemplate redisTemplate) {
        return new MessageListenerAdapter(new MessageListener() {
            @Override
            public void onMessage(Message message, byte[] pattern) {
                byte[] channel = message.getChannel();
                byte[] body = message.getBody();

                String cacheName = (String) redisTemplate.getStringSerializer().deserialize(channel);
                LocalRedisCache cache = (LocalRedisCache) cacheManager.getCache(cacheName);
                if (cache == null) {
                    return;
                }

                UpdateMessage updateMessage = (UpdateMessage) redisTemplate.getValueSerializer().deserialize(body);
                cache.sub( updateMessage);
            }
        });
    }

}

 

 

到些,实现基本完成,其他相关简单代码,参考附件。

分享到:
评论
1 楼 无忧何往 2019-01-18  
实际用了,发现有点问题;
1.redis发布的message应该带上cache name,这样方便获取;如果是按照文中用topic,也就是channel来获取每次都是一样的
2.构造RedisCacheManager时候应该传入EcacheManager
3.应该是自己实现一个cachemanager然后持有二级缓存(redisCacheManager)和一级缓存(EcacheManager)的引用;这样耦合不紧密

相关推荐

Global site tag (gtag.js) - Google Analytics