Displaying #cassandra-dev/2016-10-21.log:

Fri Oct 21 00:25:26 2016  mstepura:Joined the channel
Fri Oct 21 00:26:15 2016  clohfink:Joined the channel
Fri Oct 21 00:37:21 2016  kohlisankalp:Joined the channel
Fri Oct 21 01:33:51 2016  mstepura:Joined the channel
Fri Oct 21 01:53:08 2016  mstepura:Joined the channel
Fri Oct 21 02:33:50 2016  zznate:one of these days ill do a commit accross mulitple versions and everything will go right.
Fri Oct 21 02:40:21 2016  exlt:I tried to review a small patch set earlier this evening and decided the merge and changes madness needed a fresh cup of coffee in in the morning ;)
Fri Oct 21 02:41:29 2016  exlt:my git foo will get stronger, however
Fri Oct 21 03:22:32 2016  mstepura:Joined the channel
Fri Oct 21 03:29:33 2016  mstepura:Joined the channel
Fri Oct 21 03:30:23 2016  mstepura:Joined the channel
Fri Oct 21 03:31:09 2016  mstepura:Joined the channel
Fri Oct 21 03:32:03 2016  mstepura:Joined the channel
Fri Oct 21 03:32:50 2016  mstepura:Joined the channel
Fri Oct 21 03:33:37 2016  mstepura:Joined the channel
Fri Oct 21 03:34:26 2016  mstepura:Joined the channel
Fri Oct 21 03:37:48 2016  mstepura:Joined the channel
Fri Oct 21 04:02:05 2016  mstepura:Joined the channel
Fri Oct 21 04:28:56 2016  mstepura:Joined the channel
Fri Oct 21 04:59:52 2016  mstepura:Joined the channel
Fri Oct 21 05:03:15 2016  mstepura:Joined the channel
Fri Oct 21 05:04:04 2016  mstepura:Joined the channel
Fri Oct 21 05:04:53 2016  mstepura:Joined the channel
Fri Oct 21 05:05:39 2016  mstepura:Joined the channel
Fri Oct 21 05:06:29 2016  mstepura:Joined the channel
Fri Oct 21 05:07:19 2016  mstepura:Joined the channel
Fri Oct 21 05:08:09 2016  mstepura:Joined the channel
Fri Oct 21 05:08:58 2016  mstepura:Joined the channel
Fri Oct 21 05:09:46 2016  mstepura:Joined the channel
Fri Oct 21 05:10:33 2016  mstepura:Joined the channel
Fri Oct 21 05:11:21 2016  mstepura:Joined the channel
Fri Oct 21 05:12:13 2016  mstepura:Joined the channel
Fri Oct 21 05:12:58 2016  mstepura:Joined the channel
Fri Oct 21 05:13:53 2016  mstepura:Joined the channel
Fri Oct 21 05:14:39 2016  mstepura:Joined the channel
Fri Oct 21 05:15:00 2016  charger:Joined the channel
Fri Oct 21 05:15:28 2016  mstepura:Joined the channel
Fri Oct 21 05:16:13 2016  mstepura:Joined the channel
Fri Oct 21 05:17:03 2016  mstepura:Joined the channel
Fri Oct 21 05:17:48 2016  mstepura:Joined the channel
Fri Oct 21 05:18:37 2016  mstepura:Joined the channel
Fri Oct 21 05:24:21 2016  kohlisankalp:Joined the channel
Fri Oct 21 05:29:34 2016  kredd:Joined the channel
Fri Oct 21 05:46:35 2016  mstepura:Joined the channel
Fri Oct 21 07:15:09 2016  spodkowinski:Joined the channel
Fri Oct 21 07:52:02 2016  minimarcel:Joined the channel
Fri Oct 21 09:13:18 2016  gila:Joined the channel
Fri Oct 21 10:33:11 2016  minimarcel:Joined the channel
Fri Oct 21 10:37:09 2016  minimarcel___:Joined the channel
Fri Oct 21 11:02:18 2016  cassandra097:Joined the channel
Fri Oct 21 11:28:02 2016  minimarcel___:Joined the channel
Fri Oct 21 11:44:53 2016  minimarcel___:Joined the channel
Fri Oct 21 11:44:55 2016  kvaster:Joined the channel
Fri Oct 21 11:59:51 2016  gila:Joined the channel
Fri Oct 21 12:17:34 2016  kvaster:Joined the channel
Fri Oct 21 12:22:01 2016  gila:Joined the channel
Fri Oct 21 12:26:44 2016  jmckenzie:Joined the channel
Fri Oct 21 12:33:04 2016  kvaster:Joined the channel
Fri Oct 21 12:57:49 2016  clohfink:Joined the channel
Fri Oct 21 13:20:41 2016  mal``:Joined the channel
Fri Oct 21 13:35:52 2016  minimarcel___:Joined the channel
Fri Oct 21 13:38:30 2016  clohfink:Joined the channel
Fri Oct 21 13:39:22 2016  minimarcel____:Joined the channel
Fri Oct 21 13:59:09 2016  nickmbailey:Joined the channel
Fri Oct 21 14:05:23 2016  jameshowe:Joined the channel
Fri Oct 21 14:06:17 2016  jameshowe:#cassandra is always pretty quiet so I thought I'd try asking here
Fri Oct 21 14:06:30 2016  jameshowe:Is it usual to only require around eight concurrent requests to start getting these timeouts:
Fri Oct 21 14:06:40 2016  jameshowe:Cassandra timeout during write query at consistency SERIAL (3 replica were required but only 0 acknowledged the write)
Fri Oct 21 14:06:47 2016  jameshowe:Cassandra timeout during read query at consistency SERIAL (3 responses were required but only 0 replica responded)
Fri Oct 21 14:07:02 2016  jameshowe:I'm on v2.2
Fri Oct 21 14:08:16 2016  ptnapoleon:LWT requests? On the same partition? It doesn't sound unbelievable
Fri Oct 21 14:09:03 2016  jameshowe:I don't know whether that's comforting or not
Fri Oct 21 14:09:20 2016  ptnapoleon:I know LWT performs poorly when under high contention for the same partition
Fri Oct 21 14:09:30 2016  ptnapoleon:I dont know what the metric for "high" is thoug
Fri Oct 21 14:30:17 2016  marcuse_:Joined the channel
Fri Oct 21 14:46:17 2016  jameshowe:I was hoping for something a little more helpful than "yeah, they're really bad" :(
Fri Oct 21 14:48:09 2016  driftx:let me ask a different question, to what end are you using serial?
Fri Oct 21 14:50:12 2016  driftx:and what is your non-lwt (aka, not serial) CL?
Fri Oct 21 14:52:07 2016  jameshowe:billing/business transactions
Fri Oct 21 14:52:42 2016  jameshowe:it was someone's bright idea to build everything on Cassandra as soon as LWT were added
Fri Oct 21 14:54:38 2016  jameshowe:because SQL database are too difficult to globally distibute
Fri Oct 21 14:56:03 2016  driftx:no, I mean why specifically do you need to use serial?
Fri Oct 21 14:56:22 2016  jameshowe:because the operations must be fully serialised
Fri Oct 21 14:56:33 2016  driftx:what is your CL?
Fri Oct 21 14:56:39 2016  jameshowe:what does CL mean?
Fri Oct 21 14:56:44 2016  driftx:consistency level
Fri Oct 21 14:57:01 2016  jameshowe:... SERIAL
Fri Oct 21 14:57:07 2016  jameshowe:for these timeouts
Fri Oct 21 14:57:09 2016  driftx:you don't understand serial.
Fri Oct 21 14:57:25 2016  jameshowe:there's SERIAL or LOCAL_SERIAL
Fri Oct 21 14:57:39 2016  driftx:what is your actual CL though? Because serial is separate.
Fri Oct 21 14:58:03 2016  jameshowe:QUORUM
Fri Oct 21 14:58:24 2016  driftx:then you probably don't need serial
Fri Oct 21 14:59:26 2016  jameshowe:how do you ensure that you're changing the version of a row that you think you are, and there wasn't another write in between?
Fri Oct 21 15:00:03 2016  driftx:use an appropriate CL and LWT
Fri Oct 21 15:00:11 2016  driftx:do you know what serial does?
Fri Oct 21 15:01:39 2016  jameshowe:an LWT write is run with either SERIAL or LOCAL_SERIAL, equivalent to QUORUM or LOCAL_QUORUM
Fri Oct 21 15:01:55 2016  driftx:serial does nothing on a write
Fri Oct 21 15:04:22 2016  jameshowe:hang on, with the Java Driver, consistencyLevel is ONE and serialConsistencyLevel is SERIAL and I think these LWT writes are prepared with QUORUM
Fri Oct 21 15:04:32 2016  jameshowe:is that not right?
Fri Oct 21 15:05:06 2016  driftx:uh, by default? I don't know.
Fri Oct 21 15:05:22 2016  thobbs:Joined the channel
Fri Oct 21 15:05:26 2016  driftx:but if your CL is quorum, you don't need serial, unless something is going to read at ONE
Fri Oct 21 15:05:50 2016  jameshowe:with those settings, the timeout message is "write query at consistency SERIAL"
Fri Oct 21 15:06:19 2016  jameshowe:reads are usually at ONE, but switch to SERIAL if we're trying to deal with an LWT timeout
Fri Oct 21 15:07:31 2016  driftx:I'm not sure you understand this well enough
Fri Oct 21 15:07:44 2016  jameshowe:it would seem so
Fri Oct 21 15:08:36 2016  jameshowe:I guess Datastax documentation is very misleading
Fri Oct 21 15:08:42 2016  jameshowe:what should be happening then?
Fri Oct 21 15:09:38 2016  jameshowe:we need fully serialised writes and we need to know if they succeeded, but reads are a lot more frequent and are allowed to see stale data
Fri Oct 21 15:09:46 2016  jameshowe:in Java, with v2.2
Fri Oct 21 15:10:48 2016  jameshowe:Is it the case that serialConsistencyLevel should be QUORUM, and the one you prepare the statement with is ignored?
Fri Oct 21 15:12:31 2016  jameshowe:I've just tried doing that: Invalid consistency for conditional update. Must be one of SERIAL or LOCAL_SERIAL
Fri Oct 21 15:13:06 2016  jameshowe:Which doesn't match with what you've been telling me.
Fri Oct 21 15:13:42 2016  driftx:if you're ok with reads seeing stale data, I'm not sure what you're even doing.
Fri Oct 21 15:14:07 2016  driftx:but it sounds like just quorum is all you need, heh
Fri Oct 21 15:14:08 2016  jameshowe:*most reads
Fri Oct 21 15:14:40 2016  driftx:anyway, try #datastax-drivers for driver questions
Fri Oct 21 15:14:56 2016  jameshowe:when you say "quorum is all you need" do you mean no LWT?#
Fri Oct 21 15:15:20 2016  driftx:maybe.
Fri Oct 21 15:15:32 2016  jameshowe:how do you ensure that you're changing the version of a row that you think you are, and there wasn't another write in between?
Fri Oct 21 15:16:44 2016  driftx:that's when you need LWT
Fri Oct 21 15:16:54 2016  jameshowe:which are SERIAL
Fri Oct 21 15:17:12 2016  driftx:anyway, this is off-topic here (see the guidelines) and I'm a bad person for helping, heh
Fri Oct 21 15:17:31 2016  thobbs:there are two consistency levels for LWTs
Fri Oct 21 15:17:39 2016  thobbs:the serial_consistency_level and the consistency_level
Fri Oct 21 15:17:42 2016  lukerb:Joined the channel
Fri Oct 21 15:17:43 2016  jameshowe:well i've been in #cassandra asking this for a week and have had no response
Fri Oct 21 15:18:00 2016  thobbs:this explains it pretty well: http://datastax.github.io/python-driver/api/cassandra/query.html#cassandra.query.Statement.serial_consistency_level
Fri Oct 21 15:18:05 2016  jameshowe:now a dev's telling me i don't undertsand how LWT works, but won't tell me how they do work :(
Fri Oct 21 15:18:34 2016  driftx:I don't understand the semantics you want if reading stale data is ok.
Fri Oct 21 15:18:36 2016  jameshowe:Yes, those docs describe exactly what I thought
Fri Oct 21 15:19:36 2016  jameshowe:So now we've established why I'm doing a SERIAL write, should I not expect that I should be able to do more than eight of them concurrently on a normal production cluster deployment?
Fri Oct 21 15:19:56 2016  thobbs:by concurrently, you mean all contending for the same partition?
Fri Oct 21 15:20:19 2016  jameshowe:i'm not sure whether they are the same partition or not
Fri Oct 21 15:20:28 2016  thobbs:well, you won't have any problems if they're not
Fri Oct 21 15:20:41 2016  thobbs:LWTs aren't great for highly contended updates
Fri Oct 21 15:20:50 2016  jameshowe:let's assume that they're not - what might be wrong?
Fri Oct 21 15:21:11 2016  thobbs:my guess is that they are
Fri Oct 21 15:22:18 2016  thobbs:if you're getting timeouts and they're _not_ contending for the same partition, either your cluster is overloaded or it's debugging time
Fri Oct 21 15:23:26 2016  jameshowe:I'll see you next week when I've got a debugger connected to Cassandra
Fri Oct 21 15:24:34 2016  thobbs:adding debug logging is probably a better approach, fwiw
Fri Oct 21 15:27:17 2016  jameshowe:Say someone's changing the money owed for something. That write has to be fully serialised and they need to know whether it succeeded or not. Day-to-day monitoring and websites and such are fine to not update for a few hours, as long as the person making the write operation knows that no further writes can overwrite their change by accident, and that the single read at end of quarter can see it.
Fri Oct 21 15:27:41 2016  jameshowe:hence writes at SERIAL/QUORUM with most reads at ONE.
Fri Oct 21 15:29:48 2016  jameshowe:thobbs: debug logging in Cassandra you mean?
Fri Oct 21 15:30:40 2016  lukerb:Left the channel
Fri Oct 21 15:31:00 2016  thobbs:right
Fri Oct 21 15:31:19 2016  jameshowe:thanks for your time
Fri Oct 21 15:31:35 2016  jameshowe:oh, btw is there a public log for #cassandra anywhere?
Fri Oct 21 15:31:40 2016  wadey:Joined the channel
Fri Oct 21 15:32:09 2016  ptnapoleon:No, only -dev is logged
Fri Oct 21 15:33:26 2016  jameshowe:Left the channel
Fri Oct 21 15:42:33 2016  jameshowe:Joined the channel
Fri Oct 21 15:43:00 2016  jameshowe:Sorry, when you say "partition" you mean the set of CQL "rows" that share a partition key, right?
Fri Oct 21 15:43:10 2016  ptnapoleon:yes
Fri Oct 21 15:55:43 2016  mstepura:Joined the channel
Fri Oct 21 16:15:32 2016  RobertBirnie:Joined the channel
Fri Oct 21 17:13:14 2016  kohlisankalp:Joined the channel
Fri Oct 21 17:59:37 2016  zznate:jameshowe: this preso from summit in Sept. has some excellent details on LWT https://www.youtube.com/watch?v=wcxQM3ZN20c
Fri Oct 21 19:03:36 2016  kohlisankalp:Joined the channel
Fri Oct 21 19:21:42 2016  mstepura:Joined the channel
Fri Oct 21 19:31:36 2016  kohlisankalp:Joined the channel
Fri Oct 21 20:07:08 2016  kohlisankalp:Joined the channel
Fri Oct 21 20:23:10 2016  RussSpitzer:Joined the channel
Fri Oct 21 20:31:52 2016  jmckenzie:Joined the channel
Fri Oct 21 20:44:38 2016  kohlisankalp:Joined the channel
Fri Oct 21 20:52:50 2016  kohlisankalp:Joined the channel
Fri Oct 21 21:16:33 2016  kvaster:Joined the channel
Fri Oct 21 21:17:10 2016  kvaster:Joined the channel
Fri Oct 21 21:52:20 2016  mstepura:Joined the channel
Fri Oct 21 23:06:56 2016  EnigmaCurry:Joined the channel
Fri Oct 21 23:06:57 2016  gila:Joined the channel
Fri Oct 21 23:06:57 2016  wadey:Joined the channel
Fri Oct 21 23:06:57 2016  zznate:Joined the channel
Fri Oct 21 23:06:57 2016  pcmanus:Joined the channel
Fri Oct 21 23:06:57 2016  jeffj:Joined the channel
Fri Oct 21 23:06:57 2016  JoeyI-:Joined the channel
Fri Oct 21 23:06:57 2016  aboudreault:Joined the channel
Fri Oct 21 23:06:57 2016  normanm:Joined the channel
Fri Oct 21 23:06:57 2016  thorkild:Joined the channel
Fri Oct 21 23:06:57 2016  driftx:Joined the channel
Fri Oct 21 23:06:57 2016  tjake:Joined the channel
Fri Oct 21 23:06:57 2016  mpenick:Joined the channel
Fri Oct 21 23:06:57 2016  carlyeks:Joined the channel
Fri Oct 21 23:06:57 2016  exlt:Joined the channel
Fri Oct 21 23:06:57 2016  urandom:Joined the channel
Fri Oct 21 23:06:57 2016  snazy:Joined the channel
Fri Oct 21 23:06:57 2016  mambocab:Joined the channel
Fri Oct 21 23:06:57 2016  CassBotJr:Joined the channel
Fri Oct 21 23:06:57 2016  charliek:Joined the channel
Fri Oct 21 23:06:57 2016  adamcm:Joined the channel
Fri Oct 21 23:06:57 2016  adamholmberg:Joined the channel
Fri Oct 21 23:06:57 2016  jfarrell_:Joined the channel
Fri Oct 21 23:06:57 2016  mstepura:Joined the channel
Fri Oct 21 23:07:20 2016  RussSpitzer:Joined the channel
Fri Oct 21 23:07:20 2016  vans01:Joined the channel
Fri Oct 21 23:07:20 2016  kishkaru_:Joined the channel
Fri Oct 21 23:07:20 2016  Ron__C___:Joined the channel
Fri Oct 21 23:07:20 2016  blambov:Joined the channel
Fri Oct 21 23:07:20 2016  miketra:Joined the channel
Fri Oct 21 23:07:20 2016  gdusbabek:Joined the channel
Fri Oct 21 23:07:20 2016  iamaleksey:Joined the channel
Fri Oct 21 23:07:20 2016  aweisberg:Joined the channel
Fri Oct 21 23:07:27 2016  beobal:Joined the channel
Fri Oct 21 23:07:27 2016  marcuse_:Joined the channel
Fri Oct 21 23:07:34 2016  RobertBirnie:Joined the channel
Fri Oct 21 23:07:34 2016  lqid:Joined the channel
Fri Oct 21 23:07:42 2016  mebigfatguy:Joined the channel
Fri Oct 21 23:07:42 2016  mikewall-:Joined the channel
Fri Oct 21 23:08:02 2016  kohlisankalp:Joined the channel
Fri Oct 21 23:08:02 2016  mal``:Joined the channel
Fri Oct 21 23:08:02 2016  blerer:Joined the channel
Fri Oct 21 23:08:02 2016  rsmags_:Joined the channel
Fri Oct 21 23:08:02 2016  beckje01:Joined the channel
Fri Oct 21 23:08:02 2016  mck_sw:Joined the channel
Fri Oct 21 23:08:02 2016  rxo:Joined the channel
Fri Oct 21 23:08:02 2016  yukim:Joined the channel
Fri Oct 21 23:08:02 2016  paulomotta:Joined the channel
Fri Oct 21 23:08:02 2016  rustyrazorblade:Joined the channel
Fri Oct 21 23:08:02 2016  thobbs_irccloud:Joined the channel
Fri Oct 21 23:08:02 2016  ostefano_:Joined the channel
Fri Oct 21 23:08:02 2016  stephenc:Joined the channel
Fri Oct 21 23:08:02 2016  jrwest_:Joined the channel
Fri Oct 21 23:08:02 2016  zanson:Joined the channel
Fri Oct 21 23:08:08 2016  jasobrown:Joined the channel
Fri Oct 21 23:08:08 2016  ifesdjeen:Joined the channel
Fri Oct 21 23:08:08 2016  wei:Joined the channel
Fri Oct 21 23:08:08 2016  bdeggleston:Joined the channel
Fri Oct 21 23:08:08 2016  jbellis_:Joined the channel
Fri Oct 21 23:08:08 2016  stef1927:Joined the channel
Fri Oct 21 23:08:08 2016  eribeiro:Joined the channel
Fri Oct 21 23:08:08 2016  jeromatron:Joined the channel
Fri Oct 21 23:08:08 2016  ptnapoleon:Joined the channel
Fri Oct 21 23:12:59 2016  cassci:Joined the channel
Fri Oct 21 23:13:00 2016  krut:Joined the channel
Fri Oct 21 23:16:29 2016  garren:Joined the channel
Fri Oct 21 23:16:30 2016  garren:Joined the channel
Fri Oct 21 23:21:10 2016  ryansvihla:Joined the channel
Fri Oct 21 23:28:42 2016  jkni:Joined the channel
Fri Oct 21 23:31:03 2016  techntoke__:Joined the channel
Fri Oct 21 23:32:50 2016  Nate75Sanders:Joined the channel
Fri Oct 21 23:34:38 2016  mstepura:Joined the channel
Fri Oct 21 23:38:27 2016  joseph_arnstein_:Joined the channel
Fri Oct 21 23:39:37 2016  cory___:Joined the channel
Fri Oct 21 23:40:47 2016  felowe:Joined the channel

Comments