`
yvonxiao
  • 浏览: 77290 次
  • 性别: Icon_minigender_1
  • 来自: 上海
文章分类
社区版块
存档分类
最新评论

基于kryo序列化方案的memcached-session-manager多memcached节点配置

阅读更多

更新:

现在做分布式解决方案一般都是基于cookie不开启HttpSession来实现的,当时是因为单机系统已经开发好了后面为了方便水平扩展而选择了memcached-session-manager,这几天碰巧又用到了这个,发现最新的稳定版本1.8.3已经很方便的可以使用kryo来做序列化了,这里贴一个我现在用的单机Memcached的配置

 

<Manager className="de.javakaffee.web.msm.MemcachedBackupSessionManager"
    memcachedNodes="n1:127.0.0.1:11211"
    sticky="false"
    lockingMode="auto"
    memcachedProtocol="binary"
    sessionBackupAsync="false"
    requestUriIgnorePattern=".*\.(ico|png|gif|jpg|css|js)$"
    transcoderFactoryClass="de.javakaffee.web.msm.serializer.kryo.KryoTranscoderFactory"
    />   

 

新版本不需要以前的一些比如cglib-nodep-2.2.jar等包了,具体可自行参考官网的配置

 

另外说一句,本来是打算用tomcat-redis-session-manager的,后来发现这货实现的方案还是有bug的,比如在spring mvc中使用RedirectAttributes.addFlashAttribute会发现这方法无效,换了它官方说明的Persistence Policies都不行,也不打算替他修正了,直接换久经考验的方式。

----------------------------------------------------------分界线-----------------------------------------------------------------

 

  上次基于Java IO的序列化方案配置了memcached-session-manager,但是性能不好,现在先简单配置成基于kryo的

<Context path="/mobilemail" docBase="D:\webapp\WebRoot" reloadable="true">
<Manager 
className= "de.javakaffee.web.msm.MemcachedBackupSessionManager" 
memcachedNodes= "n1:192.168.112.1:11211,n2:192.168.112.2:11211" 
sticky="false" 
lockingMode="auto" 
requestUriIgnorePattern= ".*\.(png|gif|jpg|css|js)$"  
sessionBackupAsync= "false"
memcachedProtocol="binary" 
copyCollectionsForSerialization="true" 
transcoderFactoryClass="de.javakaffee.web.msm.serializer.kryo.KryoTranscoderFactory"  
customConverter="org.claros.commons.CustomKryoRegistration"
 />
</Context>

 

 注意customConverter配置的com.test.serializer.CustomKryoRegistration这个类是自己写的类,是用来注册一些特殊类型的,同时自己负责对这些类型的序列化,可以放在项目里,也可以单独打个jar包,把以后遇到的所有复杂类型都在这里即可。项目里遇到的kryo没有注册到的类就是session里的java.util.concurrent.ConcurrentHashMap这个类,反序列化的时候本来可以做得更高效些,这里只是为了演示,直接用kryo自带的MapSerializer来反序列化

 

package com.test.serializer;
import java.util.concurrent.ConcurrentHashMap;

import com.esotericsoftware.kryo.Kryo;
import com.esotericsoftware.kryo.serialize.MapSerializer;

import de.javakaffee.web.msm.serializer.kryo.KryoCustomization;

public class CustomKryoRegistration implements KryoCustomization {
	public void customize(Kryo kryo) {
		kryo.register(ConcurrentHashMap.class, new MapSerializer(kryo));
	}
}

 

 

 

感兴趣的同学请参见msm-kryo-serializer-1.4.0.jar注册wicket的MiniMap的流程(de.javakaffee.web.msm.serializer.kryo.WicketMiniMapRegistration里对MiniMap使用kryo-serializers-0.8.jar中的de.javakaffee.kryoserializers.wicket.MiniMapSerializer 单独处理)
 
要使用kryo进行序列化需额外添加kryo的jar包,我使用的是kryo-1.04,所需要的jar包在附件里。添加依赖的时候请注意不要引起jar包冲突,比较常见的是使用cglib和所依赖的asm的冲突,我的解决办法是留下kryo所需要的asm-3.3.1.jar,删去其他版本的asm,再将原先项目里的cglib换成cglib-nodep-2.2.jar。

 

分享到:
评论
3 楼 godo121 2012-06-22  
cheng888qi 写道
你好!
最近也在配置这个但是遇到了问题,web框架中用到了spring security,
我试过了像LZ这样配置,但是 还是出现
Mar 23, 2011 6:24:35 PM de.javakaffee.web.msm.MemcachedBackupSessionManager loadFromMemcached
WARNING: Could not load session with id C9593682F286B6054292A348B27F4CE7-n1 from memcached.
com.esotericsoftware.kryo.SerializationException: Unable to deserialize object of type: java.util.concurrent.ConcurrentHashMap
        at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:584)
        at com.esotericsoftware.kryo.ObjectBuffer.readObject(ObjectBuffer.java:209)
        at de.javakaffee.web.msm.serializer.kryo.KryoTranscoder.deserializeAttributes(KryoTranscoder.java:257)
        at de.javakaffee.web.msm.TranscoderService.deserializeAttributes(TranscoderService.java:157)
        at de.javakaffee.web.msm.TranscoderService.deserialize(TranscoderService.java:114)
        at de.javakaffee.web.msm.MemcachedBackupSessionManager.loadFromMemcached(MemcachedBackupSessionManager.java:929)
        at de.javakaffee.web.msm.MemcachedBackupSessionManager.findSession(MemcachedBackupSessionManager.java:519)
        at org.apache.catalina.connector.Request.isRequestedSessionIdValid(Request.java:2138)
        at org.apache.catalina.connector.CoyoteAdapter.parseSessionCookiesId(CoyoteAdapter.java:592)
        at org.apache.catalina.connector.CoyoteAdapter.postParseRequest(CoyoteAdapter.java:509)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:291)
        at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:849)
        at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:583)
        at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:454)
        at java.lang.Thread.run(Thread.java:662)
Caused by: com.esotericsoftware.kryo.SerializationException: Unable to deserialize object of type: org.springframework.security.core.context.SecurityContextImpl
        at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:562)
        at com.esotericsoftware.kryo.serialize.MapSerializer.readObjectData(MapSerializer.java:129)
        at com.esotericsoftware.kryo.Serializer.readObject(Serializer.java:61)
        at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:580)
        ... 14 more
Caused by: com.esotericsoftware.kryo.SerializationException: Serialization trace:
authorities (com.fz.cloudsync.security.user.SecurityUser)
principal (org.springframework.security.authentication.UsernamePasswordAuthenticationToken)
authentication (org.springframework.security.core.context.SecurityContextImpl)
        at com.esotericsoftware.kryo.serialize.FieldSerializer.readObjectData(FieldSerializer.java:230)
        at com.esotericsoftware.kryo.serialize.ReferenceFieldSerializer.readObjectData(ReferenceFieldSerializer.java:81)
        at com.esotericsoftware.kryo.serialize.FieldSerializer.readObjectData(FieldSerializer.java:212)
        at com.esotericsoftware.kryo.serialize.ReferenceFieldSerializer.readObjectData(ReferenceFieldSerializer.java:81)
        at com.esotericsoftware.kryo.serialize.FieldSerializer.readObjectData(FieldSerializer.java:212)
        at com.esotericsoftware.kryo.serialize.ReferenceFieldSerializer.readObjectData(ReferenceFieldSerializer.java:81)
        at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:557)
        ... 17 more
Caused by: java.lang.ClassCastException: org.springframework.security.core.authority.GrantedAuthorityImpl cannot be cast to java.lang.Comparable
        at java.util.TreeMap.put(TreeMap.java:542)
        at java.util.TreeSet.add(TreeSet.java:238)
        at com.esotericsoftware.kryo.serialize.CollectionSerializer.readObjectData(CollectionSerializer.java:113)
        at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:557)
        at de.javakaffee.kryoserializers.UnmodifiableCollectionsSerializer.readObjectData(UnmodifiableCollectionsSerializer.java:84)
        at com.esotericsoftware.kryo.serialize.FieldSerializer.readObjectData(FieldSerializer.java:212)
        ... 23 more

能帮我解决一下吗?谢谢!

http://www.oschina.net/question/3270_20436
2 楼 yvonxiao 2011-04-16  
cheng888qi 写道
你好!
最近也在配置这个但是遇到了问题,web框架中用到了spring security,
我试过了像LZ这样配置,但是 还是出现
Mar 23, 2011 6:24:35 PM de.javakaffee.web.msm.MemcachedBackupSessionManager loadFromMemcached
WARNING: Could not load session with id C9593682F286B6054292A348B27F4CE7-n1 from memcached.
com.esotericsoftware.kryo.SerializationException: Unable to deserialize object of type: java.util.concurrent.ConcurrentHashMap
        at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:584)
        at com.esotericsoftware.kryo.ObjectBuffer.readObject(ObjectBuffer.java:209)
        at de.javakaffee.web.msm.serializer.kryo.KryoTranscoder.deserializeAttributes(KryoTranscoder.java:257)
        at de.javakaffee.web.msm.TranscoderService.deserializeAttributes(TranscoderService.java:157)
        at de.javakaffee.web.msm.TranscoderService.deserialize(TranscoderService.java:114)
        at de.javakaffee.web.msm.MemcachedBackupSessionManager.loadFromMemcached(MemcachedBackupSessionManager.java:929)
        at de.javakaffee.web.msm.MemcachedBackupSessionManager.findSession(MemcachedBackupSessionManager.java:519)
        at org.apache.catalina.connector.Request.isRequestedSessionIdValid(Request.java:2138)
        at org.apache.catalina.connector.CoyoteAdapter.parseSessionCookiesId(CoyoteAdapter.java:592)
        at org.apache.catalina.connector.CoyoteAdapter.postParseRequest(CoyoteAdapter.java:509)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:291)
        at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:849)
        at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:583)
        at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:454)
        at java.lang.Thread.run(Thread.java:662)
Caused by: com.esotericsoftware.kryo.SerializationException: Unable to deserialize object of type: org.springframework.security.core.context.SecurityContextImpl
        at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:562)
        at com.esotericsoftware.kryo.serialize.MapSerializer.readObjectData(MapSerializer.java:129)
        at com.esotericsoftware.kryo.Serializer.readObject(Serializer.java:61)
        at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:580)
        ... 14 more
Caused by: com.esotericsoftware.kryo.SerializationException: Serialization trace:
authorities (com.fz.cloudsync.security.user.SecurityUser)
principal (org.springframework.security.authentication.UsernamePasswordAuthenticationToken)
authentication (org.springframework.security.core.context.SecurityContextImpl)
        at com.esotericsoftware.kryo.serialize.FieldSerializer.readObjectData(FieldSerializer.java:230)
        at com.esotericsoftware.kryo.serialize.ReferenceFieldSerializer.readObjectData(ReferenceFieldSerializer.java:81)
        at com.esotericsoftware.kryo.serialize.FieldSerializer.readObjectData(FieldSerializer.java:212)
        at com.esotericsoftware.kryo.serialize.ReferenceFieldSerializer.readObjectData(ReferenceFieldSerializer.java:81)
        at com.esotericsoftware.kryo.serialize.FieldSerializer.readObjectData(FieldSerializer.java:212)
        at com.esotericsoftware.kryo.serialize.ReferenceFieldSerializer.readObjectData(ReferenceFieldSerializer.java:81)
        at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:557)
        ... 17 more
Caused by: java.lang.ClassCastException: org.springframework.security.core.authority.GrantedAuthorityImpl cannot be cast to java.lang.Comparable
        at java.util.TreeMap.put(TreeMap.java:542)
        at java.util.TreeSet.add(TreeSet.java:238)
        at com.esotericsoftware.kryo.serialize.CollectionSerializer.readObjectData(CollectionSerializer.java:113)
        at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:557)
        at de.javakaffee.kryoserializers.UnmodifiableCollectionsSerializer.readObjectData(UnmodifiableCollectionsSerializer.java:84)
        at com.esotericsoftware.kryo.serialize.FieldSerializer.readObjectData(FieldSerializer.java:212)
        ... 23 more

能帮我解决一下吗?谢谢!

你配置了customConverter么,这个要自己去写反序列化的类的,java.lang.ClassCastException:org.springframework.security.core.authority.GrantedAuthorityImpl cannot be cast to java.lang.Comparable,我给的例子是做个例子来演示的,你要根据你的实际情况自己处理
1 楼 cheng888qi 2011-03-24  
你好!
最近也在配置这个但是遇到了问题,web框架中用到了spring security,
我试过了像LZ这样配置,但是 还是出现
Mar 23, 2011 6:24:35 PM de.javakaffee.web.msm.MemcachedBackupSessionManager loadFromMemcached
WARNING: Could not load session with id C9593682F286B6054292A348B27F4CE7-n1 from memcached.
com.esotericsoftware.kryo.SerializationException: Unable to deserialize object of type: java.util.concurrent.ConcurrentHashMap
        at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:584)
        at com.esotericsoftware.kryo.ObjectBuffer.readObject(ObjectBuffer.java:209)
        at de.javakaffee.web.msm.serializer.kryo.KryoTranscoder.deserializeAttributes(KryoTranscoder.java:257)
        at de.javakaffee.web.msm.TranscoderService.deserializeAttributes(TranscoderService.java:157)
        at de.javakaffee.web.msm.TranscoderService.deserialize(TranscoderService.java:114)
        at de.javakaffee.web.msm.MemcachedBackupSessionManager.loadFromMemcached(MemcachedBackupSessionManager.java:929)
        at de.javakaffee.web.msm.MemcachedBackupSessionManager.findSession(MemcachedBackupSessionManager.java:519)
        at org.apache.catalina.connector.Request.isRequestedSessionIdValid(Request.java:2138)
        at org.apache.catalina.connector.CoyoteAdapter.parseSessionCookiesId(CoyoteAdapter.java:592)
        at org.apache.catalina.connector.CoyoteAdapter.postParseRequest(CoyoteAdapter.java:509)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:291)
        at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:849)
        at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:583)
        at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:454)
        at java.lang.Thread.run(Thread.java:662)
Caused by: com.esotericsoftware.kryo.SerializationException: Unable to deserialize object of type: org.springframework.security.core.context.SecurityContextImpl
        at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:562)
        at com.esotericsoftware.kryo.serialize.MapSerializer.readObjectData(MapSerializer.java:129)
        at com.esotericsoftware.kryo.Serializer.readObject(Serializer.java:61)
        at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:580)
        ... 14 more
Caused by: com.esotericsoftware.kryo.SerializationException: Serialization trace:
authorities (com.fz.cloudsync.security.user.SecurityUser)
principal (org.springframework.security.authentication.UsernamePasswordAuthenticationToken)
authentication (org.springframework.security.core.context.SecurityContextImpl)
        at com.esotericsoftware.kryo.serialize.FieldSerializer.readObjectData(FieldSerializer.java:230)
        at com.esotericsoftware.kryo.serialize.ReferenceFieldSerializer.readObjectData(ReferenceFieldSerializer.java:81)
        at com.esotericsoftware.kryo.serialize.FieldSerializer.readObjectData(FieldSerializer.java:212)
        at com.esotericsoftware.kryo.serialize.ReferenceFieldSerializer.readObjectData(ReferenceFieldSerializer.java:81)
        at com.esotericsoftware.kryo.serialize.FieldSerializer.readObjectData(FieldSerializer.java:212)
        at com.esotericsoftware.kryo.serialize.ReferenceFieldSerializer.readObjectData(ReferenceFieldSerializer.java:81)
        at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:557)
        ... 17 more
Caused by: java.lang.ClassCastException: org.springframework.security.core.authority.GrantedAuthorityImpl cannot be cast to java.lang.Comparable
        at java.util.TreeMap.put(TreeMap.java:542)
        at java.util.TreeSet.add(TreeSet.java:238)
        at com.esotericsoftware.kryo.serialize.CollectionSerializer.readObjectData(CollectionSerializer.java:113)
        at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:557)
        at de.javakaffee.kryoserializers.UnmodifiableCollectionsSerializer.readObjectData(UnmodifiableCollectionsSerializer.java:84)
        at com.esotericsoftware.kryo.serialize.FieldSerializer.readObjectData(FieldSerializer.java:212)
        ... 23 more

能帮我解决一下吗?谢谢!

相关推荐

Global site tag (gtag.js) - Google Analytics