Java Mailing List Archive

http://www.junlu.com/

Home » users-digest.tomcat »

users Digest 12 Mar 2013 22:28:58 -0000 Issue 11291

users-digest-help

2013-03-12


Author LoginPost Reply

users Digest 12 Mar 2013 22:28:58 -0000 Issue 11291

Topics (messages 240296 through 240317)

Re: Tomcat jdbc pool connection failover
 240296 by: amit shah
 240297 by: Christopher Schultz

Re: AJP suddenly Stopps acting: ajp on 7009 and 9009 : connections keept open
 240298 by: David Kumar
 240302 by: Christopher Schultz

Re: [OT] Console when running as a service.
 240299 by: Christopher Schultz

Re: date format in Last-modified header
 240300 by: Christopher Schultz

Re: Tomcat as a service: system tray?
 240301 by: Sam Takoy
 240303 by: Sam Takoy
 240308 by: Caldarale, Charles R
 240310 by: Konstantin Kolinko
 240317 by: André Warnier

tomcat 6.0.35 in production maintaince
 240304 by: fachhoch
 240309 by: Caldarale, Charles R
 240311 by: Shanti Suresh
 240312 by: Leo Donahue - RDSA IT
 240313 by: Christopher Schultz

Can Tomcat 8 snapshots be published to Maven?
 240305 by: Nick Williams
 240306 by: Mark Thomas
 240307 by: Mark Thomas
 240316 by: Nick Williams

Deadlock when using jetty 8 JDBCSessionManager and Tomcat 7 JDBC Connector
 240314 by: Colin Ingarfield
 240315 by: Mark Thomas

Administrivia:

---------------------------------------------------------------------
To post to the list, e-mail: users@(protected)
To unsubscribe, e-mail: users-digest-unsubscribe@(protected)
For additional commands, e-mail: users-digest-help@(protected)

----------------------------------------------------------------------


Attachment: users_240296.eml (zipped)
I understand that unplanned outages are rare but we need a high
availability solution which does not involve manual intervention.


On Mon, Mar 11, 2013 at 9:05 PM, Shanti Suresh <shanti@(protected):

> On Mon, Mar 11, 2013 at 11:00 AM, André Warnier <aw@(protected):
>
> >
> > amit shah wrote:
> >
> >> Apparently we use Oracle as our data store while the solution you
> >> mentioned
> >> is for mysql.
> >>
> >
> > And did you check if the Oracle JDBC driver provides such an option ?
> >
> >
> >
> >> Is there a way tomcat can provide a notification after all its retries
> to
> >> the database server have failed? This way at least I can create a new
> >> connection pool configured to a different database server in order to
> >> simulate a automatic connection failover from the application side. I
> >> understand that this step would require time for the retries and
> creation
> >> of new pool.
> >>
> >
> > I am not a Java specialist, but I would imagine that at least your
> > application would get some kind of I/O exception when that happens on a
> > read or a write.
> > Can you not catch that and do whatever you need to then ?
> > After all, even if the driver itself handles the primary case, there
> would
> > still be a possibility that even your backup database server would also
> be
> > unreachable, no ?
> >
> >
> How often does an unplanned outage occur? It would seem drastic if an
> unplanned database server outage occurs often.
> For planned server outages, usually, the failover is done out-of-band with
> Tomcat. Please correct me if I am wrong.
> Some simple options I can think of include:
> * DNS mapping - map the DNS name of the Oracle server to the new IP address
> of the second server - make sure that "networkaddress.cache.ttl=1800" or
> something other than "-1" in the jre/lib/java.security file
> * a loadbalancer listing both database servers with one inactive - then you
> mark the secondary server as active, and the primary as inactive during
> maintenance of the primary database server.
>
> Thanks.
>
>               -Shanti
>
> >
> >
> >> On Mon, Mar 11, 2013 at 6:15 PM, Daniel Mikusa <dmikusa@(protected)>
> >> wrote:
> >>
> >> On Mar 11, 2013, at 12:52 AM, amit shah wrote:
> >>>
> >>> Hello,
> >>>>     I would like to know if the tomcat jdbc pool (7.0.34+) provides
> >>>> connection failover capabilities i.e. to transparently close all the
> >>>> open
> >>>> database connections and switch to a another database server on an
> >>>> planned/unplanned database server outage event.
> >>>>     I read through the tomcat documentation but didn't find any
> >>>>
> >>> details
> >>>
> >>>> related to this. If this feature is not supported are there any
> >>>>
> >>> recommended
> >>>
> >>>> alternatives and any future plans to add this feature to the jdbc
> pool?
> >>>>
> >>> You might want to see if your JDBC driver will handle this for you. I
> >>> know that some do, like MySQL [1].
> >>>
> >>
>

Attachment: users_240297.eml (zipped)
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Amit,

On 3/12/13 7:54 AM, amit shah wrote:
> I am using Oracle. Oracle JDBC Driver provides the Oracle
> Universal Connection Pool (UCP) which includes this
> feature<http://docs.oracle.com/cd/E11882_01/java.112/e16548/fstconfo.htm>of
>
>
connection failover but since we use tomcat jdbc connection pool we
> cannot use UCP.

Why not?

> Also UCP has lot of synchronized code which leads to blocking
> threads and less concurrency support.
>
> Let me know your suggestions/thoughts.

I'm thinking that a low-performance fail-over is preferable to a
zero-performance non-fail-over.

- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iEYEAREIAAYFAlE/VEoACgkQ9CaO5/Lv0PDnvgCfdimagPAHXpTmNex8ZlWlb5Te
7BsAoIUiGWLZ2LTWOb4r7LBV/XciWj6u
=RvDu
-----END PGP SIGNATURE-----


Attachment: users_240298.eml (zipped)
Hey,

thanks..

I downloaded jmxsh... I keep you guy up to date..

thanks...

Mit freundlichen Grüßen
David Kumar
Softwareentwickler, B. Sc.
Abteilung Infotech - Interaktiv
TELESTAR-DIGITAL GmbH
Am Weiher 14
D-56766 Ulmen

http://www.telestar.de/




-----Ursprüngliche Nachricht-----
Von: André Warnier [mailto:aw@(protected)]
Gesendet: Dienstag, 12. März 2013 09:54
An: Tomcat Users List
Betreff: Re: AJP suddenly Stopps acting: ajp on 7009 and 9009 : connections keept open

David Kumar wrote:
> Hey,
>
> we are still having that issue.
> But we could manage to figure out some more stuff.
> We made a Tomcat and Java update since that time we had our problem a few times again, also we did some reconfiguration with connectors etc.
> The last 2 times we where able to see, that both tomcat by them self where alive. Just ajp on both where dead. We couldn't make a connection either trough 7009 nor 9009. An with our openFiles trick we found a lot of close_wait again, e.g. 200 for 9009. I left the second tomcat on this state for a few ours just to see, what happens. The count of 200 connection with close_wait was kept until a reboot of the tomcat.

Instead of rebooting Tomcat, try to force the Tomcat JVM to do a Major Garbage Collection.
There are a number of tools that allow to do that.
One command-line one which I found practical is jmxsh, here :
http://code.google.com/p/jmxsh/

If when you do a Major GC, these CLOSE_WAIT connections disappear, you will have learned
something about their origin.
And if then - without restarting Tomcat - you can connect again via the AJP ports, you'll
have learned something else.

Go do it and report.


> I would say with some of our reconfiguration we managed to stop increasing connections. But still we are not sure why our ajp connections dying..
>
> Here is our connector out of Server.xml:
>
>   <Connector port="9009" protocol="AJP/1.3" redirectPort="9443" maxThreads="200" connectionTimeout="600000" />
>
>
> worker.properties:
>
> worker.tomcatX.host=localhost
> worker.tomcatX.type=ajp13
> worker.tomcatX.fail_on_status=404
> worker.tomcatX.lbfactor=1
> worker.tomcatX.ping_timeout=1000
> worker.tomcatX.ping_mode=A
> worker.tomcatX.socket_timeout=10
> worker.tomcatX.connection_pool_timeout=600
>
>
> worker.tomcat1.reference=worker.tomcatX
> worker.tomcat1.port=7009
>
> worker.tomcat2.reference=worker.tomcatX
> worker.tomcat2.port=9009
>
> worker.loadbalancer.type=lb
> worker.loadbalancer.balance_workers=tomcat1,tomcat2
>
> worker.status.type=status
>
> Hopefully one of you guys can give us a hint to fix that problem.
>
> Mit freundlichen Grüßen
> David Kumar
> Softwareentwickler, B. Sc.
> Abteilung Infotech - Interaktiv
> TELESTAR-DIGITAL GmbH
> Am Weiher 14
> D-56766 Ulmen
>
> http://www.telestar.de/
>
>
>
>
> -----Ursprüngliche Nachricht-----
> Von: David Kumar [mailto:dkumar@(protected)]
> Gesendet: Dienstag, 22. Januar 2013 07:36
> An: Tomcat Users List
> Betreff: AW: AW: AW: ajp on 7009 and 9009 : connections keept open
>
> Hey,
>
> last friday I changed our configuration to use a executor.
> Here is what I did:
>
> <Connector port="7009" protocol="AJP/1.3" redirectPort="8443" maxThreads="200" executor="active-executor" />
> <Executor name="active-executor" namePrefix="activeThread-" maxThreads="200" minSpareThreads="30" maxIdleTime="60000" />
>   <Connector port="7080" protocol="HTTP/1.1"
>           connectionTimeout="20000"
>           redirectPort="8443" executor="active-executor" />
>
> The second tomcat has same configuration besides ports..
>
> Until yesterday it worked like a charm. But at late afternoon one of the tomcats failed again..
>
> I couldn't start the garbagecollection so far..
>
> Any other ideas?
>
> Thanks
>
> Mit freundlichen Grüßen
> David Kumar
> Softwareentwickler, B. Sc.
> Abteilung Infotech - Interaktiv
> TELESTAR-DIGITAL GmbH
> Am Weiher 14
> D-56766 Ulmen
> http://www.telestar.de/
>
>
>
>
> -----Ursprüngliche Nachricht-----
> Von: David Kumar [mailto:dkumar@(protected)]
> Gesendet: Freitag, 18. Januar 2013 11:19
> An: Tomcat Users List; Tomcat Users List
> Betreff: AW: AW: AW: ( ajp on 7009 and 9009 not afs3-rmtsys): connections keept open
>
> Hey,
>
> I do that at next deployment. --> I Thursday...
>
> So far I'm trying executor for tomcat. As far I read, when I'm using connectors idle process are forced to be close..
>
>
> Mit freundlichen Grüßen
> David Kumar
> Softwareentwickler, B. Sc.
> Abteilung Infotech - Interaktiv
> TELESTAR-DIGITAL GmbH
> Am Weiher 14
> D-56766 Ulmen
>
> http://www.telestar.de/
>
>
>
>
> -----Ursprüngliche Nachricht-----
> Von: André Warnier [mailto:aw@(protected)]
> Gesendet: Freitag, 18. Januar 2013 11:10
> An: Tomcat Users List
> Betreff: Re: AW: AW: ( ajp on 7009 and 9009 not afs3-rmtsys): connections keept open
>
> David Kumar wrote:
>> Hey André,
>>
>> are you talking about running System.gc()?
> Yes.
>
>> That should be possible..
>>
>> Mit freundlichen Grüßen
>> David Kumar
>> Softwareentwickler, B. Sc.
>> Abteilung Infotech - Interaktiv
>> TELESTAR-DIGITAL GmbH
>> Am Weiher 14
>> D-56766 Ulmen
>>
>> http://www.telestar.de/
>>
>>
>>
>>
>> -----Ursprüngliche Nachricht-----
>> Von: André Warnier [mailto:aw@(protected)]
>> Gesendet: Freitag, 18. Januar 2013 10:07
>> An: Tomcat Users List
>> Betreff: Re: AW: ( ajp on 7009 and 9009 not afs3-rmtsys): connections keept open
>>
>> David,
>>  (and sorry for top-posting here)
>>
>> just to verify something.
>> Can you trigger a Major Garbage Collection at the Tomcat JVM level, at a moment when you
>> have all these connections in CLOSE_WAIT, and see if they disappear after the GC ?
>>
>> If yes, it may give a good clue about where all these CLOSE_WAITs are coming from.
>>
>>
>> David Kumar wrote:
>>> Just read this email.. :-)
>>>
>>> I figured out we are not using executor connector...
>>>
>>>
>>> Mit freundlichen Grüßen
>>> David Kumar
>>> Softwareentwickler, B. Sc.
>>> Abteilung Infotech - Interaktiv
>>> TELESTAR-DIGITAL GmbH
>>> Am Weiher 14
>>> D-56766 Ulmen
>>> http://www.telestar.de/
>>>
>>>
>>>
>>>
>>> -----Ursprüngliche Nachricht-----
>>> Von: David Kumar [mailto:dkumar@(protected)]
>>> Gesendet: Freitag, 18. Januar 2013 09:11
>>> An: Tomcat Users List
>>> Betreff: AW: ( ajp on 7009 and 9009 not afs3-rmtsys): connections keept open
>>>
>>> here you are with attachment :-)
>>>
>>>
>>> btw: in mod_jk.log I found some
>>> [Thu Jan 17 23:00:08 2013] [11196:140336689317632] [error] ajp_get_reply::jk_ajp_common.c (2055): (tomcat2) Tomcat is down or refused connection. No response has been sent to the client (yet)
>>> [Thu Jan 17 23:00:08 2013] [11196:140336689317632] [error] ajp_service::jk_ajp_common.c (2559): (tomcat2) connecting to tomcat failed.
>>>
>>>
>>> but realy just a few one...
>>>
>>>
>>> Mit freundlichen Grüßen
>>> David Kumar
>>> Softwareentwickler, B. Sc.
>>> Abteilung Infotech - Interaktiv
>>> TELESTAR-DIGITAL GmbH
>>> Am Weiher 14
>>> D-56766 Ulmen
>>>
>>> http://www.telestar.de/
>>>
>>>
>>>
>>>
>>> -----Ursprüngliche Nachricht-----
>>> Von: David Kumar
>>> Gesendet: Freitag, 18. Januar 2013 09:08
>>> An: 'Tomcat Users List'
>>> Betreff: ( ajp on 7009 and 9009 not afs3-rmtsys): connections keept open
>>>
>>> Hey,
>>>
>>> thanks for reply. I got that about the Apache configuration. Since we had our problem yesterday, again and there was no error at the apache logs I'm willing to say that is not the main problem and I have to check that, when my main problem is solved.. :-)
>>>
>>>
>>>
>>> I agree with you about wrong reporting of service. Its just shown up as afs3 because these service uses 7009 per default. But I'm using 7009 and 9009 for ajp.
>>>
>>>
>>> So doesn't this mean there is a connection problems between my Apache and the tomcats?
>>>
>>> You're right, both Webapps doing the same and are configured identically besides the ports.
>>>
>>> I'm using more than one database, but all of them are used through a database pool. If there is a bug, I think I should have found some error at my logs like no free connection or something like that. As there is no such log entry I'm willing to say that my database connections processing like they should.
>>>
>>> Basically on each tomcat there are running two services. One is a axis2 project. Our CRM is posting customer data to this webapp. This data will be persisted into a database. Depending on the information given by our CRM axis sends a email.
>>>
>>> The second one is basically a cache for our websites. We have a PIM with all our product data. These app is gathering all the data from PIM and a CMS and is merging these information together so that the data can be displayed.
>>> All the mentioned data is hold in different "cache objects". Also some communication with our ERP and some databases are made trough this app.
>>>
>>> The second app is a REST service. Information will be posted as POST or GET request to it. Most likely the responses are JSON Object.
>>>
>>> When ever one webApp is reloading (automatically or manually) itself, the result will be posted to the other tomcat/webapp as a serialized object, so the other on do not need to reload it self.
>>>
>>> I can't say how many SMB files there are, it is depending on some other stuff so it is dynamic.
>>>
>>> Attached you can find a printed list by lsof.
>>>
>>> There you can see a really strange thing. Yesterday just tomcat2 had the problem with to many open files. A few days before it was just tomcat1 having this problem.
>>>
>>> Now let my answer your question:
>>>
>>> 1. That is hard to say, I guess I have to do some more investigation on our logfiles.
>>>
>>> 2. / 3. Here is my httpd.conf:
>>> <IfModule mpm_worker_module>
>>>  ThreadLimit       25      
>>>  StartServers       2
>>>  MaxClients       150
>>>  MinSpareThreads    25
>>>  MaxSpareThreads    75
>>>  ThreadsPerChild    25
>>>  MaxRequestsPerChild  4000
>>> </IfModule>
>>>
>>> we are using worker ....
>>>
>>> And here are our tomcat connectors again:
>>> tomcat1:
>>>
>>> <Connector port="7080" protocol="HTTP/1.1" connectionTimeout="20000" redirectPort="8443"/>
>>>
>>>
>>> <Connector port="7009" protocol="AJP/1.3" redirectPort="8443"/>
>>>
>>>
>>> tomcat2:
>>>    <Connector port="9080" protocol="HTTP/1.1" connectionTimeout="20000" redirectPort="9443"/>
>>>
>>> <Connector port="9009" protocol="AJP/1.3" redirectPort="9443"/>
>>>
>>>
>>> Okay we are not using executor.. I will check that..
>>>
>>> You probably read my copy-paste error. I did copy some comments out of out server config --> Sry again.
>>>
>>> 4. we are using..
>>> 5. via a multipart message sending to the other tomcat.
>>> 6. I don't think so also because of that the connections are kept open on our ajp ports.
>>>
>>> I know that "CLOSE_WAIT" means, waiting for connections to be closed, but wondering that it is not closing..
>>>
>>>
>>> Thanks again
>>>
>>> Mit freundlichen Grüßen
>>> David Kumar
>>> Softwareentwickler, B. Sc.
>>> Abteilung Infotech - Interaktiv
>>> TELESTAR-DIGITAL GmbH
>>> Am Weiher 14
>>> D-56766 Ulmen
>>> http://www.telestar.de/
>>>
>>>
>>>
>>>
>>> -----Ursprüngliche Nachricht-----
>>> Von: Christopher Schultz [mailto:chris@(protected)]
>>> Gesendet: Donnerstag, 17. Januar 2013 18:38
>>> An: Tomcat Users List
>>> Betreff: Re: AW: AW: afs3-rmtsys: connections keept open
>>>
>>> -----BEGIN PGP SIGNED MESSAGE-----
>>> Hash: SHA256
>>>
>>> David,
>>>
>>> On 1/17/13 1:49 AM, David Kumar wrote:
>>>> I just checked /var/logs/apache2/error.logs. And found following
>>>> errors:
>>>>
>>>> [Wed Jan 16 15:14:46 2013] [error] server is within MinSpareThreads
>>>> of MaxClients, consider raising the MaxClients setting [Wed Jan 16
>>>> 15:14:56 2013] [error] server reached MaxClients setting, consider
>>>> raising the MaxClients setting
>>> So you are maxing-out your connections: you are experiencing enough
>>> load that your configuration cannot handle any more connections:
>>> requests are being queued by the TCP/IP stack and some requests may be
>>> rejected entirely depending upon the queue length of the socket.
>>>
>>> The first question to ask yourself is whether or not your hardware can
>>> take more than you have it configured to accept. For instance, if your
>>> load average, memory usage, and response time are all reasonable, then
>>> you could probably afford to raise your MaxClients setting in httpd.
>>>
>>> Note that the above has almost nothing to do with Tomcat: it only has
>>> to do with Apache httpd.
>>>
>>>> Yesterday my problem occurred about the same time.
>>> So, the problem is that Tomcat cannot handle your peak load due to a
>>> file handle limitation. IIRC, your current file handle limit for the
>>> Tomcat process is 4096.
>>>
>>>> I'm checking every five minutes how many open files there are:
>>>>
>>>> count open files started: 01-16-2013_15:10: Count: 775 count open
>>>> files started: 01-16-2013_15:15: Count: 1092
>>> Okay. lsof will help you determine how many of those are "real" files
>>> versus sockets. Limiting socket usage might be somewhat easier
>>> depending upon what your application actually does.
>>>
>>>> But maybe the afs3 connection causing the Apache error?
>>> afs3 is a red herring: you are using port 7009 for AJP communication
>>> between httpd and Tomcat and it's being reported as afs3. This has
>>> nothing to do with afs3 unless you know for a fact that your web
>>> application uses that protocol for something. I don't see any evidence
>>> that afs3 is related to your environment in the slightest. I do see
>>> every indication that you are using port 7009 yourself for AJP so
>>> let's assume that's the truth.
>>>
>>> Let's recap what your webapp(s) actually do to see if we can't figure
>>> out where all your file handles are being used. I'll assume that each
>>> Tomcat is configured (reasonably) identically, other than port numbers
>>> and such. I'll also assume that you are running the same webapp using
>>> the same (virtually) identical configuration and that nothing
>>> pathological is happening (like one process totally going crazy and
>>> making thousands of socket connections due to an application bug).
>>>
>>> First, all processes need access to stdin, stdout, stderr: that's 3
>>> file handles. Plus all shared libraries required to get the process
>>> and JVM started. Plus everything Java needs. Depending on the OS,
>>> that's about 30 or so to begin with. Then, Tomcat uses /dev/random (or
>>> /dev/urandom) plus it needs to load all of its own libraries from JAR
>>> files. There are about 25 of them, and they generally stay open. So,
>>> we're up to about 55 file handles. Don't worry: we won't be counting
>>> these things one-at-a-time for long. Next, Tomcat has two <Connector>s
>>> defined with default connection sizes. At peak load, they will both be
>>> maxed-out at 200 connections each for a total of 402 file handles (1
>>> bind file handle + 200 file handles for the connections * 2
>>> connectors). So, we're up to 457.
>>>
>>> Now, onto your web application. You have to count the number of JAR
>>> files that your web application provides: each one of those likely
>>> consumes another file handle that will stay open. Does your webapp use
>>> a database? If so, do you use a connection pool? How big is the
>>> connection pool? Do you have any leaks? If you use a connection pool
>>> and have no leaks, then you can add 'maxActive' file handles to our
>>> running count. If you don't use a connection pool, then you can add
>>> 400 file handles to your count, because any incoming request on either
>>> of those two connectors could result in a database connection. (I
>>> highly recommend using a connection pool if you aren't already).
>>>
>>> Next, you said this:
>>>
>>>> Both of the tomcats are "synchronising" them self. The send some
>>>> serialized objects via http to each other.
>>> So the webapps make requests to each other? How? Is there a limit to
>>> the number of connections directly from one Tomcat to another? If not,
>>> then you can add another 400 file handles because any incoming
>>> connection could trigger an HTTP connection to the other Tomcat. (What
>>> happens if an incoming client connection causes a connection to the
>>> other Tomcat... will that Tomcat ever call-back to the first one and
>>> set-up a communication storm?).
>>>
>>>> And both of them getting some file from SMB shares.
>>> How many files? Every file you open consumes a file handle. If you
>>> close the file, you can reduce your fd footprint, but if you keep lots
>>> of files open...
>>>
>>> If you have a dbcp with size=50 and you limit your cross-Tomcat
>>> connections to, say another 50 and your webapp uses 50 JAR files then
>>> you are looking at 600 or so file handles required to run your webapp
>>> under peak load, not including files that must be opened to satisfy a
>>> particular request.
>>>
>>> So the question is: where are all your fds going? Use lsof to
>>> determine what they are being used for.
>>>
>>> Some suggestions:
>>>
>>> 1. Consider the number of connections you actually need to be able to
>>> handle: for both connectors. Maybe you don't need 200 possible
>>> connections for your HTTP connector.
>>>
>>> 2. Make sure your MaxClients in httpd matches make sense with what
>>> you've got in Tomcat's AJP connector: you want to make sure that you
>>> have enough connections available from httpd->Tomcat that you aren't
>>> making users wait. If you're using prefork MPM that means that
>>> MaxClients should be the same as your <Connector>'s maxThreads setting
>>> (or, better yet, use an <Executor>).
>>>
>>> 3. Use an <Executor>. Right now, you might allocate up to 400 threads
>>> to handle connections from both AJP and HTTP. Maybe you don't need
>>> that. You can share request-processing threads by using an <Executor>
>>> and have both connectors share the same pool.
>>>
>>> 4. Use a DBCP. Just in case you aren't.
>>>
>>> 5. Check to see how you are communicating Tomcat-to-Tomcat: you may
>>> have a problem where too many connections are being opened.
>>>
>>> 6. Check to make sure you don't have any resource leaks: JDBC
>>> connections that aren't closed, files not being closed, etc. etc.
>>> Check to make sure you are closing files that don't need to be open
>>> after they are read.
>>>
>>>> But I can't imagine that might be the problem? I'm wondering why
>>>> the tcp connections with state "CLOSE_WAIT" doesn't get closed.
>>> http://en.wikipedia.org/wiki/Transmission_Control_Protocol
>>>
>>> - -chris
>>> -----BEGIN PGP SIGNATURE-----
>>> Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
>>> Comment: GPGTools - http://gpgtools.org
>>> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
>>>
>>> iEYEAREIAAYFAlD4NvsACgkQ9CaO5/Lv0PC4EACfURhDENZPf28HDIazwPqAqri5
>>> KqYAni9AOSQZVIdsBtQLoEfDcYkpuf7f
>>> =dEDY
>>> -----END PGP SIGNATURE-----
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: users-unsubscribe@(protected)
>>> For additional commands, e-mail: users-help@(protected)
>>>
>>>
>>>
>>> ------------------------------------------------------------------------
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: users-unsubscribe@(protected)
>>> For additional commands, e-mail: users-help@(protected)
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@(protected)
>> For additional commands, e-mail: users-help@(protected)
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@(protected)
>> For additional commands, e-mail: users-help@(protected)
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@(protected)
> For additional commands, e-mail: users-help@(protected)
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@(protected)
> For additional commands, e-mail: users-help@(protected)
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@(protected)
> For additional commands, e-mail: users-help@(protected)
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@(protected)
> For additional commands, e-mail: users-help@(protected)
>
>


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@(protected)
For additional commands, e-mail: users-help@(protected)



Attachment: users_240302.eml (zipped)
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

David,

On 3/12/13 6:58 AM, David Kumar wrote:
> We are using Apache 2.2.16 on Debian. Therefore MaxRequestWorkers
> is called MaxClients, isn't it?

That depends upon the MPM you are using. I have Debian-based systems
with both prefork and worker MPMs configured (separately).

> Currently it is set to 312. So if we have two tomcats with 200
> thread each MaxClients is to low? I either should reduce the Thread
> at the connector or increase MaxClients?

If you have MaxClients=312, then your Tomcat configuration should be
at least that big, even if you have two back-end Tomcats. What happens
if one of your Tomcats dies (or gets a 404 LOL)? All your traffic will
go to the other Tomcat: that means 312 connections to a single
back-end node.

So you either need to adjust your MaxClients/MaxRequestWorkers /down/
so that a single Tomcat can handle those connections, or you need to
increase your maxThreads on Tomcat to handle the potential client load
from httpd.

If you have multiple httpd instances, then you need to multiple
everything by N: MaxClients * N <= maxThreads. I would set maxThreads
to be a bit bigger, anyway, because then you can do nice things like
test to see if a Tomcat is available outside of httpd. You don't want
MaxClients == maxThreads and then not be able to ping your server due
to thread starvation.

Finally, if you have more than 8192 possible incoming connections from
all your httpd instances, you may have to adjust your maxConnections
setting (default is 8192 for APR and 10000 for NIO... default is
maxThreads for BIO which is the only thing that makes any sense at all).

- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iEYEAREIAAYFAlE/U3gACgkQ9CaO5/Lv0PD1XACdG0l/hyIBXh1CoE5Yp6XRf81P
D+kAniaCJT0KSKfSl7SSs6WLETCAB1Zx
=hqwF
-----END PGP SIGNATURE-----


Attachment: users_240299.eml (zipped)
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

André,

On 3/11/13 4:06 PM, André Warnier wrote:
> Mark Eggers wrote:
>> An alternate solution (which I've not played with in a very long
>> time) could be:
>>
>> http://poi.apache.org/
>>
>> This could remove the requirement for interacting with a desktop
>> application when dealing with some Microsoft formats.
>
> Thanks, I'll have a look at it (maybe again).

Since you have such a large volume of documents to process, you are an
ideal candidate for submitting bug reports and verifying fixes. Any
chance your employer would consider making an investment in you doing
such things? It could improve your product significantly (and save
your users licensing costs because they won't need Microsoft Office
running on their servers just to produce PDF documents).

- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iEYEAREIAAYFAlE/TtQACgkQ9CaO5/Lv0PA0sQCcCqrpIMOYn/H5yEqFoubyf5rW
aUwAnib3aJCe5O5KUhGPQlJWfMVpKG8g
=RZiw
-----END PGP SIGNATURE-----


Attachment: users_240300.eml (zipped)
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

André,

On 3/11/13 5:31 PM, André Warnier wrote:
> Just by coincidence, I happened to be looking at some HTTP/DAV
> output right now (from Apache's DAV), and I noticed this : ...
> <D:getcontenttype>application/pdf</D:getcontenttype>
> <lp1:creationdate>2012-12-21T09:25:53Z</lp1:creationdate>
> <lp1:getlastmodified>Fri, 21 Dec 2012 09:25:53
> GMT</lp1:getlastmodified>
> <lp1:getcontentlength>1009325</lp1:getcontentlength>
> <lp1:resourcetype/> ...

What is the XML Schema being used for the document? I suspect that
whatever the "lp1" namespace resolves to indicates that the timestamp
should be in xs:dateTime format which /does/ have that format. I dunno
if XML Schema allows you to constrain PCDATA, though...

Anyhow, consumers of XML will *expect* that all timestamps are in this
format. IMHO, it's a much better format than what HTTP specifies for
date headers, as it's (nearly) all numeric. Too bad Java's
SimpleDateFormat can neither emit nor consume dates in that format.
Fortunately, Apache commons has a drop-in replacement for
SimpleDateFormat that *does* support it.

> So there we have these 2 formats within a same HTTP response from
> the same module on the same server and about the same file, both of
> which could obviously have used the same format.

You're thinking about it in the wrong way(s).

First, HTTP only makes claims about the response headers: once the
headers are done, it's all bits after that... HTTP has no prevue there.

Second, standards are standards. Just because the HTTP date standard
is stupid doesn't mean you can just change it to something "better".
That would be ... non-standard. What we would need is a new HTTP
header field (e.g. Last-Modified-Not-Stupid) that is defined to be in,
say, ISO 8601 date format, or even xs:dateTime format or whatever. The
problem is that most HTTP-aware toolkits treat date header values all
the same -- take a look at HttpServletRequest.getDateHeader()... it
will parse a date header using the HTTP standard. We'd have to add a
HttpServletRequest.getNonStupidDateHeader() method, etc.

Unfortunately, HTTP is pretty much stuck forever.

XML has its own rules: xs:dateTime. It's appropriate to use them
because the document being returned is XML.

> Why make it simple when complicated will do ?

Complaining that the commonly-used XML date format doesn't agree with
the HTTP date format is like complaining that Microsoft Excel doesn't
store dates in ISO 8601 date format but instead as Julian dates in a
binary document format.

- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iEYEAREIAAYFAlE/TNwACgkQ9CaO5/Lv0PBaigCgi77IYLBmeDp+HW3FirF65H2I
C+8AoIIx7I9C6XlWRSn1AHL1s4Sq062j
=Y4y6
-----END PGP SIGNATURE-----


Attachment: users_240301.eml (zipped)
Actually, I download the .zip version of Tomcat. I installed the service using the "service install" command. So now I'm wondering, given my current situation, how to activate the system tray option.

(OK, I'll be collecting the typos as you suggested.)

Thanks!


________________________________
From: André Warnier <aw@(protected)>
To: Tomcat Users List <users@(protected)>
Sent: Tuesday, March 12, 2013 4:44 AM
Subject: Re: Tomcat as a service: system tray?

Sam Takoy wrote:
> Hi,
>
> This is related to the questions that I asked yesterday and got such insightful responses (thanks!).
>
> If I am running Tomcat as a Windows service, is it possible to control it through a System Tray icon?
>

Yes, but why don't you try it ?  It is really easy to download and install Tomcat as a Service on any Windows workstation, and you'll get the system tray icon automatically.

> (By the way, I don't know where to report irrelevant typos in the documentation, but http://tomcat.apache.org/tomcat-7.0-doc/windows-service-howto.html says "system try".)
>
Unless you want to submit patches - which are always welcome but do require some preparatory work - probably the best thing to do would be to collect a list of such typos, and when you've found a dozen or so, post the list here, with precise references like above.  Someone might then pick up the list and go through all of them in one go.


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@(protected)
For additional commands, e-mail: users-help@(protected)

Attachment: users_240303.eml (zipped)
Actually, I download the .zip version of Tomcat. I installed the service using the "service install" command. So now I'm wondering, given my current situation, how to activate the system tray option.

(OK, I'll be collecting the typos as you suggested.)

Thanks!


________________________________
From: André Warnier <aw@(protected)>
To: Tomcat Users List <users@(protected)>
Sent: Tuesday, March 12, 2013 4:44 AM
Subject: Re: Tomcat as a service: system tray?

Sam Takoy wrote:
> Hi,
>
> This is related to the questions that I asked yesterday and got such insightful responses (thanks!).
>
> If I am running Tomcat as a Windows service, is it possible to control it through a System Tray icon?
>

Yes, but why don't you try it ?  It is really easy to download and install Tomcat as a Service on any Windows workstation, and you'll get the system tray icon automatically.

> (By the way, I don't know where to report irrelevant typos in the documentation, but http://tomcat.apache.org/tomcat-7.0-doc/windows-service-howto.html says "system try".)
>
Unless you want to submit patches - which are always welcome but do require some preparatory work - probably the best thing to do would be to collect a list of such typos, and when you've found a dozen or so, post the list here, with precise references like above.  Someone might then pick up the list and go through all of them in one go.


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@(protected)
For additional commands, e-mail: users-help@(protected)

Attachment: users_240308.eml (zipped)
> From: Sam Takoy [mailto:sam.takoy@(protected)]
> Subject: Re: Tomcat as a service: system tray?

Don't top-post.

> I installed the service using the "service install" command. So now I'm wondering,
> given my current situation, how to activate the system tray option.

Read the doc:

http://commons.apache.org/proper/commons-daemon//procrun.html

Use the //MS or //MR directive.

- Chuck


THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers.



Attachment: users_240310.eml (zipped)
2013/3/12 Sam Takoy <sam.takoy@(protected)>:
> Hi,
>
> This is related to the questions that I asked yesterday and got such insightful responses (thanks!).
>
> If I am running Tomcat as a Windows service, is it possible to control it through a System Tray icon?
>

Tomcat7.exe //MS

http://commons.apache.org/proper/commons-daemon/procrun.html

Tomcat windows installer creates a shortcut for it in "Startup" group
in Windows menu, but on latest versions of Windows that app should
actually be started in "run as administrator" mode to be useful.


> (By the way, I don't know where to report irrelevant typos in the documentation, but http://tomcat.apache.org/tomcat-7.0-doc/windows-service-howto.html says "system try".)

Online Tomcat 7 documentation is integrated with comments.apache.org
If you scroll to the bottom of that page you can leave a comment
there.

Best regards,
Konstantin Kolinko


Attachment: users_240317.eml (zipped)
Sam Takoy wrote:
> Actually, I download the .zip version of Tomcat. I installed the service using the "service install" command. So now I'm wondering, given my current situation, how to activate the system tray option.
>
Sam, sorry.
I was under the impression that when you install Tomcat as a Windows Service, it
automatically creates a system tray icon. But now that I think further about it, I may
have been confusing this with Apache httpd (which I am sure does install a system tray
icon). It has been a while since I last installed a Windows-based Tomcat.
Anyway, if Tomcat doesn't, I apologise and take back my imprudent answer.


Attachment: users_240304.eml (zipped)
I am using tomcat  6.0.35 in production. I am running my web application  
on this , It   uses spring, wicket, hibernate and lot of other
technologies, problem is after one month of running my application I get out
of memory error perm gen space could not serialize mybject... .
I am using wicket framework which   serializes lot of stuff in memory I
think it is serializing lot of stuff and not enough memory is available to
serialize .

After restart the server is good for approx one month.

How can I detect in advance that my app is using up all available memory ?
I am not a server administrator or system admin, I am just a developer and
right now managing my production servers.
This out of memory perm gen space  error could this be because of some
memory leak ?

can I run gc once in week on running tomcat  should we do this for server
good health , earlier our deployment team used to deploy this and we never
had out f memory issue.
is there any maintenance necessary for production servers if so what are
those ?













--
Sent from the Tomcat - User mailing list archive at Nabble.com.


Attachment: users_240309.eml (zipped)
> From: fachhoch [mailto:fachhoch@(protected)]
> Subject: tomcat 6.0.35 in production maintaince

> How can I detect in advance that my app is using up all available memory ?

Monitor the JVM with any of a myriad of tools, such as VisualVM.

http://wiki.apache.org/tomcat/FAQ/Memory
http://wiki.apache.org/tomcat/FAQ/Troubleshooting_and_Diagnostics

> This out of memory perm gen space error could this be because of some
> memory leak ?

Pretty much guaranteed that one or more of your webapps is leaking.

http://wiki.apache.org/tomcat/MemoryLeakProtection

> can I run gc once in week on running tomcat

Pointless; the JVM is doing that automatically.

> earlier our deployment team used to deploy this and we never
> had out of memory issue.

So what did they break with the most recent deployment?

> is there any maintenance necessary for production servers if so what
> are those ?

Health monitoring is always useful.

http://wiki.apache.org/tomcat/FAQ/Monitoring
http://wiki.apache.org/tomcat/FAQ/Performance_and_Monitoring

- Chuck


THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers.



Attachment: users_240311.eml (zipped)
On Tue, Mar 12, 2013 at 1:48 PM, Caldarale, Charles R <
Chuck.Caldarale@(protected):

>
> > is there any maintenance necessary for production servers if so what
> > are those ?
>
> Health monitoring is always useful.
>
> http://wiki.apache.org/tomcat/FAQ/Monitoring
> http://wiki.apache.org/tomcat/FAQ/Performance_and_Monitoring
>
> - Chuck
>

As Chuck pointed out, find out what's causing the leak.

Some commands that might help include:

This command will print the PermGen memory usage in MB:
${JRE_HOME}//bin/jstat -gcold ${TOMCAT_PID} | /usr/bin/tail -1 | awk -F' '
'{print $2/1024}'

This command will give you a helpful snapshot of what's occupying PermGen:
${JRE_HOME}/bin/jmap -permstat ${TOMCAT_PID}

Thanks,
           -Shanti

Attachment: users_240312.eml (zipped)
>-----Original Message-----
>From: Caldarale, Charles R [mailto:Chuck.Caldarale@(protected)]
>Subject: RE: tomcat 6.0.35 in production maintaince
>
>> From: fachhoch [mailto:fachhoch@(protected)]
>> Subject: tomcat 6.0.35 in production maintaince
>
>> How can I detect in advance that my app is using up all available memory ?
>
>Monitor the JVM with any of a myriad of tools, such as VisualVM.
>
>http://wiki.apache.org/tomcat/FAQ/Memory
>http://wiki.apache.org/tomcat/FAQ/Troubleshooting_and_Diagnostics
>
>> This out of memory perm gen space error could this be because of some
>> memory leak ?
>
>Pretty much guaranteed that one or more of your webapps is leaking.
>

The default permgen space is 64MB.

Could it also run out of permgen space if you deploy many web applications that don't leak, which exhausts the default permgen memory allocation? Is the wording on this page accurate?
http://docs.oracle.com/javase/6/docs/technotes/guides/visualvm/monitor_tab.html

[quote]
PermGen. The PermGen graph displays changes in the permanent generation area over time. The permanent generation is the area of the heap where class and method objects are stored. If an application loads a very large number of classes, then the size of the permanent generation might need to be increased using the -XX:MaxPermSize option.
[/quote]

Leo


Attachment: users_240313.eml (zipped)
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Fachhoch,

On 3/12/13 10:43 AM, fachhoch wrote:
> I am using tomcat  6.0.35 in production. I am running my web
> application on this , It   uses spring, wicket, hibernate and lot
> of other technologies, problem is after one month of running my
> application I get out of memory error perm gen space could not
> serialize mybject... . I am using wicket framework which
> serializes lot of stuff in memory I think it is serializing lot of
> stuff and not enough memory is available to serialize .
>
> After restart the server is good for approx one month.
>
> How can I detect in advance that my app is using up all available
> memory ? I am not a server administrator or system admin, I am
> just a developer and right now managing my production servers. This
> out of memory perm gen space  error could this be because of
> some memory leak ?
>
> can I run gc once in week on running tomcat  should we do this for
> server good health , earlier our deployment team used to deploy
> this and we never had out f memory issue. is there any maintenance
> necessary for production servers if so what are those ?

Do you redeploy your webapp ever/often without restarting Tomcat? Or
will your app fail after ~1 month with a single deployment?

- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iEYEAREIAAYFAlE/oAMACgkQ9CaO5/Lv0PArtQCfergQpRXpG+pIdAMUP+8SRtwL
wvEAoLJsQkpNSQIv5STKayfq66xI390x
=nPFq
-----END PGP SIGNATURE-----


Attachment: users_240305.eml (zipped)
I'm experimenting with using Tomcat embedded. I need Tomcat 8.0, because I need Servlet 3.1 and WebSocket features. I can accomplish this manually I'm sure, but all of the tutorials out there for using Tomcat embedded (that I've found) show it using Maven, and frankly I prefer to use Maven when I can.

So I went on the Maven central repository and saw that the newest Tomcat on there was 7.0.37. Some projects publish SNAPSHOT artifacts from nightly trunk builds, but it appears that Tomcat does not. Two questions:

1) Is there a repository other than central that has nightly trunk snapshots published?
2) If not, is it possible to start getting nightly trunk snapshots published?

Attachment: users_240306.eml (zipped)
On 12/03/2013 17:27, Nick Williams wrote:
> I'm experimenting with using Tomcat embedded. I need Tomcat 8.0, because I need Servlet 3.1 and WebSocket features. I can accomplish this manually I'm sure, but all of the tutorials out there for using Tomcat embedded (that I've found) show it using Maven, and frankly I prefer to use Maven when I can.
>
> So I went on the Maven central repository and saw that the newest Tomcat on there was 7.0.37. Some projects publish SNAPSHOT artifacts from nightly trunk builds, but it appears that Tomcat does not. Two questions:
>
> 1) Is there a repository other than central that has nightly trunk snapshots published?
No.

> 2) If not, is it possible to start getting nightly trunk snapshots published?
Yes.

Mark


Attachment: users_240307.eml (zipped)
On 12/03/2013 17:38, Mark Thomas wrote:
> On 12/03/2013 17:27, Nick Williams wrote:
>> I'm experimenting with using Tomcat embedded. I need Tomcat 8.0,
>> because I need Servlet 3.1 and WebSocket features. I can accomplish
>> this manually I'm sure, but all of the tutorials out there for using
>> Tomcat embedded (that I've found) show it using Maven, and frankly I
>> prefer to use Maven when I can.
>>
>> So I went on the Maven central repository and saw that the newest
>> Tomcat on there was 7.0.37. Some projects publish SNAPSHOT artifacts
>> from nightly trunk builds, but it appears that Tomcat does not. Two
>> questions:
>>
>> 1) Is there a repository other than central that has nightly trunk
>> snapshots published?
> No.
>
>> 2) If not, is it possible to start getting nightly trunk snapshots
>> published?
> Yes.

Scratch that. It is a manual process so nightly snapshots won't happen.
Snapshots are still possible.

Mark


Attachment: users_240316.eml (zipped)

On Mar 12, 2013, at 12:39 PM, Mark Thomas wrote:

> On 12/03/2013 17:38, Mark Thomas wrote:
>> On 12/03/2013 17:27, Nick Williams wrote:
>>> I'm experimenting with using Tomcat embedded. I need Tomcat 8.0,
>>> because I need Servlet 3.1 and WebSocket features. I can accomplish
>>> this manually I'm sure, but all of the tutorials out there for using
>>> Tomcat embedded (that I've found) show it using Maven, and frankly I
>>> prefer to use Maven when I can.
>>>
>>> So I went on the Maven central repository and saw that the newest
>>> Tomcat on there was 7.0.37. Some projects publish SNAPSHOT artifacts
>>> from nightly trunk builds, but it appears that Tomcat does not. Two
>>> questions:
>>>
>>> 1) Is there a repository other than central that has nightly trunk
>>> snapshots published?
>> No.
>>
>>> 2) If not, is it possible to start getting nightly trunk snapshots
>>> published?
>> Yes.
>
> Scratch that. It is a manual process so nightly snapshots won't happen. Snapshots are still possible.
>

Okay. So what needs to be done to make this happen? I know nothing about publishing snapshots (or anything, for that matter) to Maven. If I can help in some way I'm willing to learn, but I imagine it takes someone with "Tomcat Authority" to publish Tomcat artifacts.

N

Attachment: users_240314.eml (zipped)
Hello,

We're using Jetty 8.1.3.v20120416 w/ JDBCSessionManager as our application
server with Tomcat 7's JDBC Connection pool 7.0.28. We've run this
particular combination in production since at least Sept 2012. (Java 6 64
bit/Ubuntu, Amazon RDS/MySQL db.)

This morning one Jetty instance stopped accepting requests and the problem
appears to be a deadlock btw a jetty connection handler thread and the conn
pool's "PoolCleaner" thread. From jstack trace:
Deadlock Detection:

Found one Java-level deadlock:
=============================

"qtp1840392480-3740":
waiting to lock Monitor@(protected)
com/mysql/jdbc/JDBC4Connection),
which is held by "PoolCleaner[2009981184:1363034108768]"
"PoolCleaner[2009981184:1363034108768]":
waiting to lock Monitor@(protected)
com/mysql/jdbc/JDBC4ResultSet),
which is held by "qtp1840392480-3740"

Once this happened all other worker threads blocked at various points
within JDBCSession manager. Obviously restarting the instance resolved the
issue for now.

How does one go about diagnosing / resolving such an issue? It's not clear
to me if either component is "at fault" here...I have the full jstack
output as well as a full heap dump of the java process, if that would help.

Regards,
Colin

Attachment: users_240315.eml (zipped)
On 12/03/2013 21:47, Colin Ingarfield wrote:
> Hello,
>
> We're using Jetty 8.1.3.v20120416 w/ JDBCSessionManager as our application
> server with Tomcat 7's JDBC Connection pool 7.0.28. We've run this
> particular combination in production since at least Sept 2012. (Java 6 64
> bit/Ubuntu, Amazon RDS/MySQL db.)
>
> This morning one Jetty instance stopped accepting requests and the problem
> appears to be a deadlock btw a jetty connection handler thread and the conn
> pool's "PoolCleaner" thread. From jstack trace:
> Deadlock Detection:
>
> Found one Java-level deadlock:
> =============================
>
> "qtp1840392480-3740":
>  waiting to lock Monitor@(protected)
> com/mysql/jdbc/JDBC4Connection),
>  which is held by "PoolCleaner[2009981184:1363034108768]"
> "PoolCleaner[2009981184:1363034108768]":
>  waiting to lock Monitor@(protected)
> com/mysql/jdbc/JDBC4ResultSet),
>  which is held by "qtp1840392480-3740"
>
> Once this happened all other worker threads blocked at various points
> within JDBCSession manager. Obviously restarting the instance resolved the
> issue for now.
>
> How does one go about diagnosing / resolving such an issue? It's not clear
> to me if either component is "at fault" here...I have the full jstack
> output as well as a full heap dump of the java process, if that would help.

Deadlocks occur when two threads attempt to acquire the same locks but
in a different order. Normally, one of the code paths is at fault. To
figure out which we need to see the full stack trace for the two threads
that are deadlocked.

Mark


©2008 junlu.com - Jax Systems, LLC, U.S.A.