Displaying #cassandra-dev/2017-03-28.log:

Tue Mar 28 00:34:16 2017  dikang:Joined the channel
Tue Mar 28 01:13:46 2017  kohlisankalp:Joined the channel
Tue Mar 28 01:25:31 2017  bdeggleston:Joined the channel
Tue Mar 28 01:36:42 2017  dikang:Joined the channel
Tue Mar 28 02:03:17 2017  adamholmberg:Joined the channel
Tue Mar 28 02:08:48 2017  EdwardCapriolo:Joined the channel
Tue Mar 28 03:04:51 2017  adamholmberg:Joined the channel
Tue Mar 28 03:20:19 2017  Vijay:Joined the channel
Tue Mar 28 03:25:16 2017  eribeiro:Joined the channel
Tue Mar 28 03:37:58 2017  dikang:Joined the channel
Tue Mar 28 03:48:51 2017  Vijay:Joined the channel
Tue Mar 28 04:05:21 2017  adamholmberg:Joined the channel
Tue Mar 28 04:08:02 2017  bbromhead:Joined the channel
Tue Mar 28 04:14:13 2017  kohlisankalp:Joined the channel
Tue Mar 28 04:14:37 2017  bbromhea_:Joined the channel
Tue Mar 28 04:37:30 2017  kohlisankalp:Joined the channel
Tue Mar 28 04:44:37 2017  kohlisankalp:Joined the channel
Tue Mar 28 05:02:48 2017  bbromhead:Joined the channel
Tue Mar 28 05:23:51 2017  bbromhea_:Joined the channel
Tue Mar 28 05:38:36 2017  makrusak:Joined the channel
Tue Mar 28 05:59:13 2017  bbromhead:Joined the channel
Tue Mar 28 06:06:41 2017  adamholmberg:Joined the channel
Tue Mar 28 07:07:56 2017  adamholmberg:Joined the channel
Tue Mar 28 07:11:01 2017  kohlisankalp:Joined the channel
Tue Mar 28 07:19:18 2017  kvaster:Joined the channel
Tue Mar 28 07:37:36 2017  spodkowinski:Joined the channel
Tue Mar 28 08:09:31 2017  adamholmberg:Joined the channel
Tue Mar 28 08:11:41 2017  kohlisankalp:Joined the channel
Tue Mar 28 09:10:21 2017  adamholmberg:Joined the channel
Tue Mar 28 09:29:54 2017  kohlisankalp:Joined the channel
Tue Mar 28 09:42:12 2017  kvaster:Joined the channel
Tue Mar 28 10:11:04 2017  adamholmberg:Joined the channel
Tue Mar 28 11:11:31 2017  adamholmberg:Joined the channel
Tue Mar 28 12:18:13 2017  jmckenzie:Joined the channel
Tue Mar 28 12:52:04 2017  makrusak:Joined the channel
Tue Mar 28 13:11:53 2017  ynazarov:Joined the channel
Tue Mar 28 13:13:11 2017  adamholmberg:Joined the channel
Tue Mar 28 13:20:10 2017  adamholmberg:Joined the channel
Tue Mar 28 13:39:54 2017  spodkowinski:Joined the channel
Tue Mar 28 14:43:28 2017  EdwardCapriolo:Joined the channel
Tue Mar 28 14:57:17 2017  ifesdjeen:Joined the channel
Tue Mar 28 15:16:36 2017  hkroger:Joined the channel
Tue Mar 28 15:43:28 2017  kohlisankalp:Joined the channel
Tue Mar 28 16:28:03 2017  Vijay_:Joined the channel
Tue Mar 28 16:41:51 2017  zaller:Joined the channel
Tue Mar 28 16:44:29 2017  kvaster:Joined the channel
Tue Mar 28 16:46:30 2017  kvaster:Joined the channel
Tue Mar 28 17:01:02 2017  dikang:Joined the channel
Tue Mar 28 17:08:07 2017  dikang_:Joined the channel
Tue Mar 28 17:25:14 2017  Vijay:Joined the channel
Tue Mar 28 17:49:55 2017  RussSpitzer:Joined the channel
Tue Mar 28 17:54:02 2017  dikang:Joined the channel
Tue Mar 28 18:17:56 2017  jeffj:what's the most straight forward way to run a dtest rolling upgrade from 2.0 -> 3.0 in a way that won't eat my laptop for 16 hours?
Tue Mar 28 18:18:17 2017  jeffj:is there some nose option that will set the upgrade matrix explicitly and only run one test icare about? i dont see it as being obvious for rolling upgrade
Tue Mar 28 18:20:15 2017  kvaster:Joined the channel
Tue Mar 28 18:21:12 2017  ptnapoleon:You want one of the upgrade through versions tests?
Tue Mar 28 18:23:10 2017  jeffj:i guess, yea - i'm just not sure how to invoke it so it only upgrades 2.0 -> 3.0 ?
Tue Mar 28 18:24:23 2017  ptnapoleon:You know it will stop through 2.1, yeah?
Tue Mar 28 18:24:48 2017  jeffj:start, 2.0, 2.1, 2.2, 3.0, done
Tue Mar 28 18:26:17 2017  ptnapoleon:We have nothing that does that, actually
Tue Mar 28 18:26:27 2017  ptnapoleon:Only 2.0 -> 2.1 -> 2.2 or 2.1 -> 2.2 -> 3.0
Tue Mar 28 18:26:39 2017  ptnapoleon:You could probably write one yourself. Does 3.0 take protocol v2?
Tue Mar 28 18:26:46 2017  jeffj:dont think it does
Tue Mar 28 18:27:17 2017  jeffj:so our tests basically rely on a persistent native proto session, which means we're bound to versions that share the same native proto
Tue Mar 28 18:27:22 2017  ptnapoleon:Yep
Tue Mar 28 18:27:32 2017  ptnapoleon:We dont change proto version throughout the test
Tue Mar 28 18:27:38 2017  jeffj:so a feature in 2.0 that was "legacy" in 2.1 and breaks in 3.0 cant be tested in the dtests we have right now
Tue Mar 28 18:27:46 2017  ptnapoleon:correct
Tue Mar 28 18:27:59 2017  ptnapoleon:is this supercolumns
Tue Mar 28 18:28:06 2017  jeffj:no
Tue Mar 28 18:28:18 2017  jeffj:#13384 - table caching params
Tue Mar 28 18:28:19 2017  CassBotJr:https://issues.apache.org/jira/browse/CASSANDRA-13384 (Patch Available; Unresolved; 3.0.x, 3.11.x): "Legacy caching options can prevent 3.0 upgrade"
Tue Mar 28 18:28:41 2017  jeffj:2.0 took raw string, 2.1 took json, 3.0 barfs if it sees raw string
Tue Mar 28 18:29:26 2017  ptnapoleon:I dont have a good suggestion.
Tue Mar 28 18:29:32 2017  ptnapoleon:fix the tests to change protocol version?
Tue Mar 28 18:29:35 2017  jeffj:that actually makes me feel better
Tue Mar 28 18:29:40 2017  jeffj:because it means i'm not as lost as i feared
Tue Mar 28 18:30:46 2017  dikang:Joined the channel
Tue Mar 28 18:54:33 2017  ifesdjeen:Joined the channel
Tue Mar 28 19:03:07 2017  kvaster:Joined the channel
Tue Mar 28 19:14:45 2017  rustyrazorblade:there's been a lot of changes to the docs recently, maybe we should update cassandra.apache.org w/ the latest?
Tue Mar 28 19:16:11 2017  iamaleksey:no ned to go through 2.2
Tue Mar 28 19:16:23 2017  iamaleksey:just make sure 2.1 is at least 2.1.9
Tue Mar 28 19:16:32 2017  iamaleksey:actually, now it needs to be latest
Tue Mar 28 19:17:04 2017  iamaleksey:@jeffj there is code in 2.1 that on startup converts old caching values to maps
Tue Mar 28 19:17:15 2017  iamaleksey:that's re #13384
Tue Mar 28 19:17:16 2017  CassBotJr:https://issues.apache.org/jira/browse/CASSANDRA-13384 (Patch Available; Unresolved; 3.0.x, 3.11.x): "Legacy caching options can prevent 3.0 upgrade"
Tue Mar 28 19:17:37 2017  iamaleksey:I've seen this issue crop up before just once, it's a JIRA duplicate
Tue Mar 28 19:18:32 2017  jeffj:it converts it to a map, but does it persist it to disk as a map?
Tue Mar 28 19:18:50 2017  iamaleksey:by converts I meant converts persistently
Tue Mar 28 19:19:16 2017  iamaleksey:I remember b/c I either reviewed or wrote it
Tue Mar 28 19:20:21 2017  iamaleksey:see SystemKeyspace.migrateCachingOptions
Tue Mar 28 19:24:53 2017  jeffj:hrm.
Tue Mar 28 19:26:30 2017  nickmbailey:Joined the channel
Tue Mar 28 19:27:51 2017  dikang:Joined the channel
Tue Mar 28 19:43:01 2017  EdwardCapriolo:Joined the channel
Tue Mar 28 19:45:09 2017  vjarnot:Joined the channel
Tue Mar 28 19:56:45 2017  nickmbailey:Joined the channel
Tue Mar 28 20:04:29 2017  Vijay:Joined the channel
Tue Mar 28 20:37:27 2017  urandom:here is a fun one (he says with tongue in cheek) anyone care to take a quick crack at all of the uses for timestamps? for example, as a thought experiment, what would happen if you used a constant for write-time (say 0)?
Tue Mar 28 20:37:46 2017  urandom:as driftx points out, deletes would be... interesting
Tue Mar 28 20:38:27 2017  urandom:seems like there are lot of places that the timestamp permeates
Tue Mar 28 20:46:40 2017  jeffj:two cells with equal timestamps are compared by cell value, right?
Tue Mar 28 20:58:46 2017  EdwardCapriolo:Yes if timestamps equal cell value is compared
Tue Mar 28 20:59:45 2017  EdwardCapriolo:urandom: In practices for cases where you are update only and do not wish to overwrite cells the timestamp can be an effective batch identifier.
Tue Mar 28 21:00:28 2017  urandom:jeffj: yeah
Tue Mar 28 21:00:37 2017  EdwardCapriolo:IE I do not want a timeuuuid to be part of my comparator/partition key because it is essentially redundant. (Though I do not know if anyone uses it that way in practice)
Tue Mar 28 21:02:36 2017  jeffj:i'm sure i've mentioned before that 2 jobs ago we used to use timestamps as leaderboard values
Tue Mar 28 21:02:52 2017  urandom:you *can* make the precedent come from the value and not the timestamp, but i suspect all sorts of things will break in bizarre ways
Tue Mar 28 21:02:57 2017  jeffj:highest score wins without read before write
Tue Mar 28 21:03:07 2017  urandom:jeffj: in cassandra?
Tue Mar 28 21:03:09 2017  jeffj:yea
Tue Mar 28 21:03:35 2017  jeffj:we'd use cassandra as store of record, and do actual sorting in redis sorted sets (with persistence turned off, and we'd just suck back out of cassandra again if we lost a redis shard)
Tue Mar 28 21:04:24 2017  jeffj:redis sorted sets = skiplists = o(lg n) for things like calculating rank (which with 10s of millions of players was pretty nice)
Tue Mar 28 21:12:43 2017  EdwardCapriolo:4 companies ago our software would only re-segment someone if it detected the segements were old. So the timestamp was a natural fit there
Tue Mar 28 21:27:42 2017  urandom:EdwardCapriolo: i'm not sure i follow, what was the batch identifier?
Tue Mar 28 21:28:51 2017  urandom:EdwardCapriolo: i mean, i assume it was a long of some kind, but what did it represent?
Tue Mar 28 21:29:01 2017  urandom:were they total ordered?
Tue Mar 28 21:29:31 2017  urandom:were you writing the same record with the same batch id?
Tue Mar 28 21:31:18 2017  EdwardCapriolo:CREATE TABLE SEGMENTS ( uid string, segment string, primary key (uid,segment) ); The TS gives be a "batch identifier" for free. Otherwise I would need to do CREATE TABLE SEGMENTS (uid string, segment string, time long, primary key(uid,segment)
Tue Mar 28 21:32:30 2017  EdwardCapriolo:Thus my etl process can blind write segments and my client, and my process con ignore the old batch of segments to be deleted later.
Tue Mar 28 21:33:24 2017  urandom:EdwardCapriolo: so you were using the writetime of a batch as the batch ID?
Tue Mar 28 21:33:48 2017  urandom:and overwriting the previous batch
Tue Mar 28 21:34:19 2017  EdwardCapriolo:Yes. In short, why write a redundant column which is essentially a copy of TS
Tue Mar 28 21:34:34 2017  urandom:gotcha
Tue Mar 28 21:34:48 2017  urandom:that's different to my (anti)proposal here
Tue Mar 28 21:35:47 2017  urandom:i'm wondering, if you wrote everything with say a timestamp of say Long.MIN_VALUE, or 0, and allowed the value to determine precedence (say, by prefixing a big endian), what would happen?
Tue Mar 28 21:36:26 2017  urandom:deletes wouldn't be possible, because no matter what the precendence of a tombstone, it'd be wrong in some way
Tue Mar 28 21:36:44 2017  urandom:either you'd never be able to write the value again, or it wouldn't delete
Tue Mar 28 21:37:24 2017  urandom:but the timestamp seems like something with pretty far reaching ramifications
Tue Mar 28 21:37:48 2017  urandom:i guess it would render certain read optimizations moot
Tue Mar 28 21:38:51 2017  urandom:I guess it would screw with tombstone GC too, wouldn't it?
Tue Mar 28 21:38:53 2017  EdwardCapriolo:urandom: I am getting ready for something to get thrown at me here :) HBASE model of keep 5 versions, enforced on compaction could work. Assuming you are sure of repaired data.
Tue Mar 28 21:39:53 2017  EdwardCapriolo:But in that model you still need something to serve as a version/time.
Tue Mar 28 21:49:27 2017  bbromhead:Joined the channel
Tue Mar 28 22:02:10 2017  bbromhead:Joined the channel
Tue Mar 28 22:26:22 2017  zaller:Joined the channel
Tue Mar 28 22:40:41 2017  bbromhead:Joined the channel
Tue Mar 28 22:42:17 2017  bbromhead:Joined the channel
Tue Mar 28 22:44:58 2017  thorkild:Joined the channel
Tue Mar 28 23:01:36 2017  adamholmberg:Joined the channel
Tue Mar 28 23:20:54 2017  Vijay:Joined the channel
Tue Mar 28 23:24:34 2017  bbromhead:Joined the channel

Comments