Displaying #traffic-server/2015-10-25.log:

Sun Oct 25 00:25:07 2015  oag:Joined the channel
Sun Oct 25 00:57:09 2015  oag:Joined the channel
Sun Oct 25 07:44:51 2015  ASFBot:Joined the channel
Sun Oct 25 08:41:56 2015  boargod2:Joined the channel
Sun Oct 25 10:36:41 2015  DjDeaf:Joined the channel
Sun Oct 25 11:15:14 2015  blattj:Joined the channel
Sun Oct 25 12:01:57 2015  masaori_:Joined the channel
Sun Oct 25 12:26:31 2015  bahumbug:Joined the channel
Sun Oct 25 12:26:31 2015  bahumbug:Joined the channel
Sun Oct 25 12:52:37 2015  blattj:Joined the channel
Sun Oct 25 12:59:45 2015  blattj:Joined the channel
Sun Oct 25 14:16:05 2015  psp:Joined the channel
Sun Oct 25 15:28:04 2015  psp:Joined the channel
Sun Oct 25 15:32:07 2015  psp1:Joined the channel
Sun Oct 25 15:55:39 2015  bahumbug:Joined the channel
Sun Oct 25 16:04:18 2015  blattj:Joined the channel
Sun Oct 25 16:05:58 2015  slx86:Joined the channel
Sun Oct 25 16:07:27 2015  slx86:Joined the channel
Sun Oct 25 20:08:38 2015  psp:Joined the channel
Sun Oct 25 20:20:41 2015  psp:Joined the channel
Sun Oct 25 20:57:26 2015  oag:Joined the channel
Sun Oct 25 21:04:27 2015  oag_:Joined the channel
Sun Oct 25 21:44:10 2015  niq:Joined the channel
Sun Oct 25 23:04:07 2015  oag:I'm having a problem with my C++ InterceptPlugin.
Sun Oct 25 23:04:40 2015  oag:It uses an AsyncHttpFetch to get some content from a back-end server and (after modifying it) returns it to the caller.
Sun Oct 25 23:05:21 2015  oag:That seems to work fine when the url I fetch is different from the incoming request, but when it's the same, it hangs.
Sun Oct 25 23:06:35 2015  oag:That sounds stupid, but it's not actually a loop because the original fetch was generated with a host header and a specific address.
Sun Oct 25 23:07:36 2015  oag:The AsyncHttpFetch resolves to a remote host, not the TS host (and it works fine when the path being fetched is different from the incoming request).
Sun Oct 25 23:14:52 2015  oag:My (naive) guess would be that TS somehow thinks that the outbound request is the same as the inbound one in progress and tells the outbound one to wait for the result of the other.
Sun Oct 25 23:24:29 2015  oag:The odd thing is that after 30s or so, the client gets an inactivity timeout generated by TS, that appears to free the AsyncHttpRequest to proceed, it gets the expected remote content, and finally, when it tries to write it to the original plugin (now deleted I suppose) there is a segfault.
Sun Oct 25 23:25:49 2015  oag:Anyone know what the cause of this is and/or how to work around it?
Sun Oct 25 23:32:31 2015  jpeach:oag: still there?
Sun Oct 25 23:32:37 2015  oag:I'm here!
Sun Oct 25 23:33:14 2015  jpeach:start debugging my turning on the http_hdr and url_rewrite diagnostic tag
Sun Oct 25 23:33:28 2015  oag:Ok..
Sun Oct 25 23:33:44 2015  jpeach:I assume that AsyncHttpFetch is backed by TSHttpFetch
Sun Oct 25 23:33:58 2015  jpeach:so the outbound plugin request will go through remapping
Sun Oct 25 23:35:24 2015  oag:Ah, ok, I got a few log entries from those tags, I think..
Sun Oct 25 23:35:46 2015  jpeach:remapping doesn't really care about where the request came from, it just looks at the request itself, so it is a good bet that the outbound is hitting the same remap rule
Sun Oct 25 23:36:40 2015  oag:The logging mentions some remap tables, all with 0 entries.
Sun Oct 25 23:37:06 2015  jpeach:is this a remap plugin?
Sun Oct 25 23:37:42 2015  jpeach:you should be able to tell whether the internal request is coming back to you'
Sun Oct 25 23:37:45 2015  oag:Well I'm not across all that exactly, what I'm doing is running TS as a forward proxy by setting the number of remaps required to 0 I think.
Sun Oct 25 23:38:21 2015  jpeach:ok, so remap_required=0
Sun Oct 25 23:38:32 2015  oag:I sort of think it isn't because when I trace it in gdb I can see that after (yep, that's the one).
Sun Oct 25 23:39:00 2015  oag:that after the client request times out, it actually goes on with the server fetch and gets the right content.
Sun Oct 25 23:39:04 2015  jpeach:so you have a global plugin that is doing an intercept?
Sun Oct 25 23:40:11 2015  oag:A GlobalPlugin installs an InterceptPlugin is how it seems to work..
Sun Oct 25 23:40:20 2015  oag:Yeah, that's right I guess.
Sun Oct 25 23:41:09 2015  jpeach:how are you hooking the original request?
Sun Oct 25 23:41:28 2015  oag:GlobalPlugin::registerHook(Plugin::HOOK_READ_REQUEST_HEADERS_PRE_REMAP);
Sun Oct 25 23:42:43 2015  jpeach:ok, are you logging anything that would tell you whether the second request is getting intercepted?
Sun Oct 25 23:43:02 2015  oag:Ah good question, let me do that and get back to you :)
Sun Oct 25 23:44:00 2015  oag:No, it only intercepts the one.
Sun Oct 25 23:45:35 2015  jpeach:I gotta step out soon, but my next avenue of investigation would be to packet trace the outbound network or send the request to a server you control
Sun Oct 25 23:46:17 2015  jpeach:historically, the HTTP client doesn't deal with chunking or content-length properly, so it might be waiting for the connection to close
Sun Oct 25 23:47:12 2015  oag:Yeah, it goes to a server I control. I was suspicious that it was waiting for a cache item to be generated by the original request, somehow confusing the two.
Sun Oct 25 23:47:26 2015  oag:But I admit I'm well out of my depth here.
Sun Oct 25 23:49:16 2015  jpeach:"works fine when the path being fetched is different from the incoming "
Sun Oct 25 23:49:42 2015  jpeach:that's weird and is not consistent with the theory I was working with
Sun Oct 25 23:51:08 2015  jpeach:could be deadlocking on the same cache key ... logging would show that with enough verbosity :-/
Sun Oct 25 23:51:24 2015  oag:That's exactly what it seems like!
Sun Oct 25 23:51:29 2015  jpeach:or you could add a query string to the path to tweak the cache key?
Sun Oct 25 23:51:41 2015  jpeach:gotta go but with check irc logs later
Sun Oct 25 23:51:57 2015  oag:I set logging on http_trans but of course all I saw was that the log s (Yes thanks so much for the help!)
Sun Oct 25 23:52:28 2015  oag:Oh that's a great idea, re query string.. at least some way to get it working.

Comments