Displaying #cassandra-dev/2016-09-28.log:

Wed Sep 28 07:00:51 2016  ASFBot:Joined the channel
Wed Sep 28 07:07:46 2016  yasuharu:Joined the channel
Wed Sep 28 07:15:45 2016  latsni:Joined the channel
Wed Sep 28 07:15:49 2016  nickmbailey:Joined the channel
Wed Sep 28 08:08:07 2016  Roman:Joined the channel
Wed Sep 28 08:08:32 2016  yasuharu:Joined the channel
Wed Sep 28 08:09:43 2016  Roman:Hello, I have a question related to CQL "CREATE TABLE ... WITH ID='xyz'". Is it possible to do so (pass the ID) also over Thrift interface? I can't figure out how.
Wed Sep 28 08:16:30 2016  nickmbailey:Joined the channel
Wed Sep 28 08:20:37 2016  minimarcel_:Joined the channel
Wed Sep 28 09:06:12 2016  Roman:Left the channel
Wed Sep 28 09:09:22 2016  yasuharu:Joined the channel
Wed Sep 28 09:17:14 2016  nickmbailey:Joined the channel
Wed Sep 28 10:10:10 2016  yasuharu:Joined the channel
Wed Sep 28 10:32:10 2016  nickmbailey:Joined the channel
Wed Sep 28 11:00:12 2016  yasuharu:Joined the channel
Wed Sep 28 11:32:50 2016  nickmbailey:Joined the channel
Wed Sep 28 12:33:42 2016  nickmbailey:Joined the channel
Wed Sep 28 12:59:42 2016  jmckenzie:Joined the channel
Wed Sep 28 13:34:25 2016  nickmbailey:Joined the channel
Wed Sep 28 13:45:36 2016  minimarcel_:Joined the channel
Wed Sep 28 13:46:48 2016  jfarrell:Joined the channel
Wed Sep 28 14:20:18 2016  tolbertam:Joined the channel
Wed Sep 28 14:31:13 2016  exlt:tjake: need your help here.. uploaded to bintray without any errors, but getting:
Wed Sep 28 14:31:16 2016  exlt:W: Failed to fetch http://www.apache.org/dist/cassandra/debian/dists/22x/main/source/Sources 404 Not Found
Wed Sep 28 14:31:19 2016  exlt:W: Failed to fetch http://www.apache.org/dist/cassandra/debian/dists/22x/main/binary-amd64/Packages 401 Unauthorized
Wed Sep 28 14:31:22 2016  exlt:W: Failed to fetch http://www.apache.org/dist/cassandra/debian/dists/22x/main/binary-i386/Packages 401 Unauthorized
Wed Sep 28 14:31:54 2016  exlt:is there something other than the `upload_bintray.sh $dir` that needs to be done?
Wed Sep 28 14:32:51 2016  tjake:You need to goto the website and hit publish
Wed Sep 28 14:33:24 2016  tjake:Also I usually kill the upload script after its done with the new artifacts
Wed Sep 28 14:35:05 2016  nickmbailey:Joined the channel
Wed Sep 28 14:35:29 2016  minimarcel_:Joined the channel
Wed Sep 28 14:35:41 2016  exlt:well, too late for killing that.. found the publish - let's see if that worked
Wed Sep 28 14:37:11 2016  exlt:must be some sort of delay - dirs are empty :(
Wed Sep 28 14:38:44 2016  tjake:Hmm, I see the files uploaded as me?
Wed Sep 28 14:39:06 2016  exlt:digging through the files in the web interface, I see they say not yet published..
Wed Sep 28 14:39:10 2016  exlt:but they are there
Wed Sep 28 14:39:24 2016  tjake:give it a few
Wed Sep 28 14:40:04 2016  exlt:yep, thanks
Wed Sep 28 14:41:03 2016  nickmbailey:Joined the channel
Wed Sep 28 14:44:33 2016  exlt:looks like it's clearing - whew. :)
Wed Sep 28 14:44:56 2016  tjake:heh
Wed Sep 28 14:47:18 2016  minimarcel_:Joined the channel
Wed Sep 28 14:55:20 2016  exlt:could a JIRA admin give me permission to release versions in JIRA?
Wed Sep 28 15:12:31 2016  iamaleksey:sec
Wed Sep 28 15:15:18 2016  iamaleksey:exlt: added you to the PMC group, not sure it's sufficient though
Wed Sep 28 15:15:21 2016  iamaleksey:give it a go?
Wed Sep 28 15:15:27 2016  iamaleksey:if not, will add you to admins
Wed Sep 28 15:15:45 2016  exlt:thanks - I'll see if I can manage versions in a few
Wed Sep 28 15:17:16 2016  exlt:indeed I can, thanks!
Wed Sep 28 15:18:20 2016  iamaleksey:woot
Wed Sep 28 15:22:47 2016  thobbs:Joined the channel
Wed Sep 28 15:38:02 2016  exlt:website update is quite opaque to me - ie. what pages constitute the "core" website? is downloads page, I suppose? is there a particular version of sphinx required? v1.3.6 from debian stable backports OK?
Wed Sep 28 15:39:30 2016  exlt:I need to also update the docs from the cassandra source, after merging a patch, but building those docs appears to be even less clear on how I'm to use jekyl
Wed Sep 28 15:40:38 2016  exlt:the makefiles help somewhat, but not sure again on version of jekyl, and if different versions behave radically different
Wed Sep 28 15:41:29 2016  thobbs:exlt: I've been using 1.3.6 sphinx, 3.0.1 jekyll
Wed Sep 28 15:42:12 2016  thobbs:exlt: did you read site/src/README already?
Wed Sep 28 15:42:20 2016  exlt:jekyll looks like 2.2.0 with no backport
Wed Sep 28 15:42:52 2016  exlt:yeah, I'm reading the README - my basic take was "read the jekyll docs"
Wed Sep 28 15:43:35 2016  thobbs:you shouldn't really need to read the jekyll docs unless you're making a big change
Wed Sep 28 15:43:38 2016  exlt:so going from a commit in the git source of cassandra, to importing somehow to svn, to building and committing what actually appears in the cassandra.a.o website? not clear at all to me
Wed Sep 28 15:44:06 2016  thobbs:so, first checkout the cassandra tag that you want to build from in a normal cassandra git repo
Wed Sep 28 15:44:31 2016  thobbs:then, you'll run a make command in the svn cassandra repo with an env var that points to your normal cassandra source
Wed Sep 28 15:44:46 2016  thobbs:that will update the file in the svn checkout
Wed Sep 28 15:44:55 2016  thobbs:files*
Wed Sep 28 15:46:19 2016  thobbs:the contents of site/src/doc are what are generated with sphinx from a normal git checkout of cassandra
Wed Sep 28 15:46:38 2016  thobbs:everything else is what's called "core" files in the readme
Wed Sep 28 15:49:41 2016  thobbs:exlt: am I making sense at all?
Wed Sep 28 15:49:46 2016  thobbs:maybe I should ask what you're trying to do
Wed Sep 28 15:50:06 2016  exlt:thanks, I'll see if I can get the download page updated, then see about src doc
Wed Sep 28 15:50:53 2016  exlt:oh, well that seems to be pulled in dynamically from somewhere, huh
Wed Sep 28 15:51:15 2016  exlt:The latest release is {{ "2.2" | full_release_link }}
Wed Sep 28 15:52:52 2016  thobbs:looks like that's coming from _plugins/release_link.rb
Wed Sep 28 15:53:00 2016  thobbs:it's a function call
Wed Sep 28 15:53:10 2016  exlt:yeah, just found the same :)
Wed Sep 28 15:55:12 2016  exlt:I'll play with jekyll here, see what the diff looks like, and go from there
Wed Sep 28 15:55:32 2016  thobbs:alright, let me know if I can help
Wed Sep 28 15:55:41 2016  thobbs:I'm not an expert, but I've messed with the docs a couple of times :)
Wed Sep 28 15:57:37 2016  exlt:_data/releases.yaml - found that specific note, so maybe this is all I need
Wed Sep 28 16:00:51 2016  exlt:yeah, this is the sort of thing that's hard to tell if it did the right thing or not.. https://paste.fscking.us/display/d57fZkAQQEFPhQAv_xFB1wftslk
Wed Sep 28 16:03:41 2016  thobbs:are you still using jekyll 2.2?
Wed Sep 28 16:04:19 2016  exlt:there are a lot of differences in html tags, ie `<p>where <code class="highlighter-rouge">...` vs `<p>where <code>..`
Wed Sep 28 16:04:19 2016  thobbs:we should probably agree to a specific version of jekyll to use and document it
Wed Sep 28 16:04:35 2016  exlt:so yeah, I think this is version specific sorts of things
Wed Sep 28 16:06:22 2016  jkni:yeah, that's definitely a Jekyll major version diff. 3.0 added rouge
Wed Sep 28 16:06:37 2016  jkni:I'm not sure how much jekyll output changes on minors/patches
Wed Sep 28 16:06:58 2016  exlt:lovely.. upgrading to jekyll 3.1.6 from debian testing is going to basically take over half of my system..
Wed Sep 28 16:07:21 2016  thobbs:exlt: you could probably install it through gem
Wed Sep 28 16:07:29 2016  exlt:there's no backport, so it's either install from source or hose my box.. should just dist-upgrade to testing and track it
Wed Sep 28 16:07:33 2016  thobbs:not that that's awesome either, but less intrusive
Wed Sep 28 16:07:37 2016  kvaster_:Joined the channel
Wed Sep 28 16:09:33 2016  jkni:Yeah, gem might be cleaner in this case
Wed Sep 28 16:09:51 2016  jkni:Jekyll also maintains Docker images which might also be helpful
Wed Sep 28 16:11:23 2016  exlt:I have to lol at that - here, install a virtual machine to change a 7 to an 8
Wed Sep 28 16:11:59 2016  jkni:Oh, I agree. I'm just not sure whether I'm lol-ing at that more or the idea of going to debian testing just to run a newer jekyll version ;)
Wed Sep 28 16:12:04 2016  exlt:oh awesome, gem wants to run as root, too lololol
Wed Sep 28 16:13:39 2016  exlt:--user-install is the ticket there
Wed Sep 28 16:14:33 2016  dikang:Joined the channel
Wed Sep 28 16:16:10 2016  mstepura:Joined the channel
Wed Sep 28 16:21:56 2016  exlt:better, but with 3.2.1 the differences are still a little odd
Wed Sep 28 16:22:18 2016  exlt:thanks for the help, this gets me a lot closer
Wed Sep 28 16:22:51 2016  kohlisankalp:Joined the channel
Wed Sep 28 16:28:28 2016  thobbs:exlt: my guess is that pcmanus is running close to the same version of ubuntu that I am, so the current docs were probably also generated with jekyll ~3.0.1
Wed Sep 28 16:29:01 2016  exlt:yeah, that's what I was going to do - see if I can downgrade
Wed Sep 28 16:29:14 2016  thobbs:gem should be pretty flexible about that
Wed Sep 28 16:29:35 2016  exlt:not sure why build is removing all of ../publish/doc/cql3 and ../publish/.htaccess
Wed Sep 28 16:30:05 2016  exlt:can revert those, so I don't mess things up, but it's odd
Wed Sep 28 16:30:26 2016  kohlisankalp:Joined the channel
Wed Sep 28 16:31:14 2016  thobbs:cql3 looks like it might be created by hand?
Wed Sep 28 16:31:36 2016  thobbs:it's just a link to old/ in the same dir
Wed Sep 28 16:53:09 2016  minimarcel_:Joined the channel
Wed Sep 28 16:57:21 2016  kohlisankalp:Joined the channel
Wed Sep 28 17:48:28 2016  steeve:Joined the channel
Wed Sep 28 18:04:51 2016  kohlisankalp:Joined the channel
Wed Sep 28 18:33:12 2016  kvaster_:Joined the channel
Wed Sep 28 18:33:28 2016  charliek:Joined the channel
Wed Sep 28 18:54:22 2016  steeve:Joined the channel
Wed Sep 28 19:08:25 2016  yasuharu:Joined the channel
Wed Sep 28 19:13:58 2016  mstepura:Joined the channel
Wed Sep 28 19:19:07 2016  kvaster_:Joined the channel
Wed Sep 28 19:21:57 2016  kvaster_:Joined the channel
Wed Sep 28 19:24:17 2016  kvaster_:Joined the channel
Wed Sep 28 19:25:04 2016  gnubeard:Joined the channel
Wed Sep 28 19:38:26 2016  gnubeard:Joined the channel
Wed Sep 28 19:39:43 2016  mstepura:Joined the channel
Wed Sep 28 19:42:26 2016  steeve_:Joined the channel
Wed Sep 28 19:58:59 2016  kohlisankalp:Joined the channel
Wed Sep 28 20:09:09 2016  yasuharu:Joined the channel
Wed Sep 28 20:19:29 2016  kvaster_:Joined the channel
Wed Sep 28 20:39:32 2016  gnubeard:Joined the channel
Wed Sep 28 20:49:51 2016  kohlisankalp:Joined the channel
Wed Sep 28 20:52:31 2016  EdwardCapriolo:Joined the channel
Wed Sep 28 20:53:54 2016  dikang:Joined the channel
Wed Sep 28 20:57:09 2016  sood:Joined the channel
Wed Sep 28 21:00:53 2016  EdwardCapriolo:All. I have been doing some digging aroung https://issues.apache.org/jira/browse/CASSANDRA-11974 . http://grepcode.com/file/repo1.maven.org/maven2/org.apache.cassandra/cassandra-all/2.1.8/org/apache/cassandra/net/OutboundTcpConnection.java#221
Wed Sep 28 21:01:21 2016  EdwardCapriolo:That looks wrong to me. Why would we be clearing the backlog? should we be clearing the list of messages we copied from the backlog?
Wed Sep 28 21:02:38 2016  kohlisan_:Joined the channel
Wed Sep 28 21:04:04 2016  kohlisa__:Joined the channel
Wed Sep 28 21:05:47 2016  thobbs:EdwardCapriolo: I'm not sure what copying you're referring to, can you point to it?
Wed Sep 28 21:06:26 2016  EdwardCapriolo:http://grepcode.com/file/repo1.maven.org/maven2/org.apache.cassandra/cassandra-all/2.1.8/org/apache/cassandra/net/OutboundTcpConnection.java#221
Wed Sep 28 21:06:47 2016  thobbs:that's the clear() call
Wed Sep 28 21:06:56 2016  EdwardCapriolo:Why clear the backlog, those are the unprocessed messages not those we tried to process
Wed Sep 28 21:07:32 2016  tjake:the connection failed in the else above so we are dropping them on the floor
Wed Sep 28 21:07:55 2016  EdwardCapriolo:cs.coalesce(backlog, drainedMessages, drainedMessageSize);
Wed Sep 28 21:08:17 2016  EdwardCapriolo:But why are we dropping the backlog and not the coalesce'd messages
Wed Sep 28 21:09:27 2016  EdwardCapriolo:Things in the backlog have not been even tried once. This disagrees with CASSANDRA-12192 with says " "Retry all internode messages once after reopening connections"
Wed Sep 28 21:09:27 2016  CassBotJr:https://issues.apache.org/jira/browse/CASSANDRA-12192 (Resolved; Fixed; 3.10): "Retry all internode messages once after reopening connections"
Wed Sep 28 21:09:58 2016  yasuharu:Joined the channel
Wed Sep 28 21:11:05 2016  tjake:closeconnection also clears the backlog
Wed Sep 28 21:11:52 2016  tjake:usually if the connect() fails you are SOL
Wed Sep 28 21:12:14 2016  thobbs:EdwardCapriolo: 12192 was mainly to address connections being closed due to failed deserialization, in which case connect() will work right away and the backlog won't be cleared
Wed Sep 28 21:12:37 2016  thobbs:not as a buffer for a node that's actually down
Wed Sep 28 21:13:34 2016  EdwardCapriolo:Interesting I will have to read that over. Two other things there 1) The loop does not always sleep so it looks like the reconnect logic can happen very fast.
Wed Sep 28 21:14:14 2016  RobertBirnie:Joined the channel
Wed Sep 28 21:17:49 2016  EdwardCapriolo:Also this is slightly different in trunk. For the life of me I can not figure out why http://grepcode.com/file/repo1.maven.org/maven2/org.apache.cassandra/cassandra-all/2.1.8/org/apache/cassandra/net/OutboundTcpConnection.java#522 does a return and not continue. I can not prove it is wrong, but with each message having a different timeout I can not quite follow why sometimes we would return early
Wed Sep 28 21:19:32 2016  EdwardCapriolo:@tjake so a general question here since the semantics is unclear to me. Since there are several cases where we just clear out the backlog is this a best effort system. IF so why is there some retry logic? I am struggling to understand a bit what it is supposed to do
Wed Sep 28 21:19:57 2016  tjake:which retry logic?
Wed Sep 28 21:20:09 2016  EdwardCapriolo:Like https://issues.apache.org/jira/browse/CASSANDRA-12192 (
Wed Sep 28 21:20:17 2016  thobbs:EdwardCapriolo: regarding the expiration logic, it's assuming that the items in the queue are in roughly chronological order
Wed Sep 28 21:20:36 2016  thobbs:so once you see one that's new enough not to be expired, the ones behind it will probably also not be expired
Wed Sep 28 21:20:39 2016  EdwardCapriolo:They may be in roughly chronological order but all ops have different timeout values
Wed Sep 28 21:20:55 2016  tjake:EdwardCapriolo: thats not a socket error
Wed Sep 28 21:20:59 2016  EdwardCapriolo:Write is 2000 and read is 5000 for example if they are ordered differently will the expire work
Wed Sep 28 21:21:00 2016  tjake:its a bad message
Wed Sep 28 21:21:34 2016  tjake:also thats on the read side not write
Wed Sep 28 21:22:04 2016  EdwardCapriolo:Also if we just clear the backlog, some verbs are considered undroppable
Wed Sep 28 21:23:15 2016  thobbs:yeah, it's just a minimal effort attempt to clear out some of the expired messages if we've fallen behind
Wed Sep 28 21:24:12 2016  thobbs:we still do the expiration check for every message before it's sent
Wed Sep 28 21:24:17 2016  EdwardCapriolo:Believe it or not under the normal case the size drops from 1025 to 0 since the expire does not cehck the size as it is expiring
Wed Sep 28 21:24:59 2016  thobbs:meaning all of the messages in the queue are expired?
Wed Sep 28 21:25:07 2016  thobbs:I'm not sure what you're saying
Wed Sep 28 21:26:05 2016  EdwardCapriolo:Meaning that the logic does not prune from 1025 to 1024 in once cycle it prunes from 1025 to 0 if they are all expired. Which is probably more or less expected.
Wed Sep 28 21:26:13 2016  thobbs:right
Wed Sep 28 21:26:51 2016  EdwardCapriolo:But not non-dropable messages, but those will get dropped if code elsewhere says .clear() :)
Wed Sep 28 21:27:35 2016  EdwardCapriolo:I am adding some unit tests and see if I can at least document the behaivor including the notes you guys gave me. TX
Wed Sep 28 21:27:51 2016  thobbs:sounds good
Wed Sep 28 21:31:09 2016  gnubeard:Joined the channel
Wed Sep 28 21:31:19 2016  steeve:Joined the channel
Wed Sep 28 21:40:21 2016  kohlisankalp:Joined the channel
Wed Sep 28 21:45:46 2016  kohlisankalp:Joined the channel
Wed Sep 28 21:54:28 2016  gnubeard:Joined the channel
Wed Sep 28 22:10:47 2016  yasuharu:Joined the channel
Wed Sep 28 22:12:34 2016  dikang:Joined the channel
Wed Sep 28 22:26:43 2016  clohfink:Joined the channel
Wed Sep 28 22:28:24 2016  kohlisankalp:Joined the channel
Wed Sep 28 22:46:27 2016  kohlisankalp:Joined the channel
Wed Sep 28 22:47:15 2016  kohlisan_:Joined the channel
Wed Sep 28 22:51:54 2016  Vijay:Joined the channel
Wed Sep 28 22:55:19 2016  gnubeard:Joined the channel
Wed Sep 28 23:11:36 2016  yasuharu:Joined the channel
Wed Sep 28 23:16:49 2016  sood:Joined the channel
Wed Sep 28 23:34:43 2016  mstepura:Joined the channel
Wed Sep 28 23:35:42 2016  mstepura:Joined the channel
Wed Sep 28 23:44:17 2016  kohlisankalp:Joined the channel
Wed Sep 28 23:53:58 2016  yasuharu:Joined the channel

Comments