Displaying #cassandra-dev/2017-05-19.log:

Fri May 19 00:10:52 2017  kohlisankalp:Joined the channel
Fri May 19 01:34:57 2017  tolbertam:Joined the channel
Fri May 19 02:09:05 2017  zaller:Joined the channel
Fri May 19 02:23:50 2017  clohfink:Joined the channel
Fri May 19 02:27:05 2017  zaller:Joined the channel
Fri May 19 02:50:06 2017  Vijay:Joined the channel
Fri May 19 02:59:47 2017  dikang:Joined the channel
Fri May 19 03:22:28 2017  Vijay:Joined the channel
Fri May 19 03:35:11 2017  tolbertam:Joined the channel
Fri May 19 03:37:26 2017  kohlisankalp:Joined the channel
Fri May 19 04:15:38 2017  kohlisankalp:Joined the channel
Fri May 19 04:53:05 2017  dikang:Joined the channel
Fri May 19 04:58:41 2017  dikang:Joined the channel
Fri May 19 05:08:18 2017  kohlisankalp:Joined the channel
Fri May 19 05:09:08 2017  kohlisankalp:Joined the channel
Fri May 19 05:09:53 2017  kohlisankalp:Joined the channel
Fri May 19 05:10:42 2017  kohlisankalp:Joined the channel
Fri May 19 05:11:28 2017  kohlisankalp:Joined the channel
Fri May 19 05:21:02 2017  dikang:Joined the channel
Fri May 19 05:35:23 2017  tolbertam:Joined the channel
Fri May 19 05:45:24 2017  Vijay_:Joined the channel
Fri May 19 05:59:42 2017  dikang:Joined the channel
Fri May 19 06:50:55 2017  spodkowinski:Joined the channel
Fri May 19 07:29:20 2017  kvaster:Joined the channel
Fri May 19 07:35:37 2017  tolbertam:Joined the channel
Fri May 19 07:36:28 2017  kvaster:Joined the channel
Fri May 19 07:41:41 2017  joaquin:Joined the channel
Fri May 19 09:18:02 2017  clohfink:Joined the channel
Fri May 19 09:34:46 2017  tolbertam:Joined the channel
Fri May 19 10:13:12 2017  iksaif:Joined the channel
Fri May 19 11:06:37 2017  clohfink:Joined the channel
Fri May 19 11:34:59 2017  tolbertam:Joined the channel
Fri May 19 13:06:36 2017  clohfink:Joined the channel
Fri May 19 13:33:22 2017  clohfink:Joined the channel
Fri May 19 13:35:12 2017  tolbertam:Joined the channel
Fri May 19 14:02:25 2017  makrusak:Joined the channel
Fri May 19 14:34:43 2017  kvaster:Joined the channel
Fri May 19 14:48:58 2017  nickmbailey:Joined the channel
Fri May 19 14:53:24 2017  kohlisankalp:Joined the channel
Fri May 19 15:15:54 2017  joaquin:Joined the channel
Fri May 19 15:20:52 2017  Vijay:Joined the channel
Fri May 19 15:35:25 2017  tolbertam:Joined the channel
Fri May 19 15:52:21 2017  nickmbailey:Joined the channel
Fri May 19 16:00:12 2017  nickmbailey:Joined the channel
Fri May 19 16:17:07 2017  EdwardCapriolo:Joined the channel
Fri May 19 16:29:31 2017  Vijay:Joined the channel
Fri May 19 16:49:23 2017  dikang:Joined the channel
Fri May 19 17:21:10 2017  zaller:Joined the channel
Fri May 19 17:36:49 2017  zaller:Joined the channel
Fri May 19 17:42:02 2017  Vijay:Joined the channel
Fri May 19 17:42:16 2017  nickmbailey:Joined the channel
Fri May 19 17:58:52 2017  Vijay:Joined the channel
Fri May 19 18:02:27 2017  Vijay_:Joined the channel
Fri May 19 18:09:46 2017  Vijay:Joined the channel
Fri May 19 18:11:34 2017  kohlisankalp:Joined the channel
Fri May 19 18:16:37 2017  kvaster:Joined the channel
Fri May 19 18:25:42 2017  kvaster:Joined the channel
Fri May 19 18:34:08 2017  kvaster:Joined the channel
Fri May 19 18:35:48 2017  Vijay:Joined the channel
Fri May 19 18:45:15 2017  urandom:Do either of these two exceptions look familiar to anyone? https://phabricator.wikimedia.org/P5460 and https://phabricator.wikimedia.org/P5460
Fri May 19 18:45:25 2017  urandom:i can't find any open tickets that seem relevant
Fri May 19 18:45:52 2017  urandom:both seem related to range tombstones, one on a read and the other on a write
Fri May 19 18:47:35 2017  urandom:damn, same link twice, the second one should be: https://phabricator.wikimedia.org/P5461
Fri May 19 18:50:04 2017  kvaster:Joined the channel
Fri May 19 18:54:39 2017  driftx:what version is that?
Fri May 19 18:56:48 2017  urandom:instaclustr's 3.7
Fri May 19 18:57:06 2017  urandom:3.7 + patches
Fri May 19 18:59:48 2017  kohlisankalp:Joined the channel
Fri May 19 18:59:48 2017  clohfink:Joined the channel
Fri May 19 19:00:05 2017  urandom:i'm doing a fair number of deletes that use inequality operators on a cluster key
Fri May 19 19:00:15 2017  kvaster:Joined the channel
Fri May 19 19:04:03 2017  driftx:¯\_(ツ)_/¯
Fri May 19 19:08:51 2017  nickmbailey:Joined the channel
Fri May 19 19:21:24 2017  nickmbailey:Joined the channel
Fri May 19 19:29:08 2017  Vijay:Joined the channel
Fri May 19 19:34:35 2017  tolbertam:Joined the channel
Fri May 19 19:35:55 2017  nickmbailey:Joined the channel
Fri May 19 20:06:48 2017  kvaster:Joined the channel
Fri May 19 20:28:54 2017  kvaster:Joined the channel
Fri May 19 20:32:48 2017  dikang:Joined the channel
Fri May 19 20:51:33 2017  makrusak:Joined the channel
Fri May 19 21:11:21 2017  nickmbailey:Joined the channel
Fri May 19 21:20:38 2017  Vijay:Joined the channel
Fri May 19 21:34:48 2017  tolbertam:Joined the channel
Fri May 19 21:36:57 2017  urandom:driftx: hrmm, i've a clue regarding the exception that occurs in the write path
Fri May 19 21:37:16 2017  urandom:i'm doing range deletes on two different clustering keys, probablistically
Fri May 19 21:37:43 2017  urandom:and the number of occurances seemed to coincide with the probability that both would occur at about the same time
Fri May 19 21:37:53 2017  urandom:so i disabled one, and that exception went away
Fri May 19 21:38:25 2017  driftx:huh.
Fri May 19 21:39:21 2017  urandom:that exception happens here: https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/db/RangeTombstoneList.java#L536
Fri May 19 21:40:08 2017  urandom:and while i am not familiar with this code, it would seem that there is an expected ordering of ranges
Fri May 19 21:40:20 2017  urandom:unexpected
Fri May 19 21:40:51 2017  urandom:i.e. start >= ends[i] ?
Fri May 19 21:45:19 2017  urandom:man, this thing is kind of a hot mess
Fri May 19 21:45:20 2017  urandom:https://grafana-admin.wikimedia.org/dashboard/db/restbase-dev?orgId=1&from=now-24h&to=now&var-node=All&var-keyspace=local_group_wikipedia_T_parsoid_html&var-table=data&var-quantile=99percentile
Fri May 19 21:45:37 2017  urandom:how is sstables/read higher than the number of sstables?
Fri May 19 21:46:19 2017  urandom:i restarted one node and it seems to have course corrected, but has since started to climb above the sstable count again
Fri May 19 21:46:47 2017  urandom:and in restarting it, for some reason, read and write latency went quite high, and hasn't come back down
Fri May 19 21:46:53 2017  urandom:weird
Fri May 19 21:47:05 2017  urandom:err...
Fri May 19 21:47:12 2017  urandom:https://grafana.wikimedia.org/dashboard/db/restbase-dev?orgId=1&from=now-24h&to=now&var-node=All&var-keyspace=local_group_wikipedia_T_parsoid_html&var-table=data&var-quantile=99percentile
Fri May 19 21:56:51 2017  driftx:that metric is wrong
Fri May 19 21:56:59 2017  driftx:it counts bf skips as sstable hits
Fri May 19 21:57:20 2017  driftx:look at blerer's email to dev today
Fri May 19 21:58:27 2017  urandom:oh?
Fri May 19 21:59:37 2017  urandom:huh
Fri May 19 22:00:06 2017  urandom:yeah, ok.
Fri May 19 22:00:33 2017  clohfink:Joined the channel
Fri May 19 22:05:29 2017  nickmbailey:Joined the channel
Fri May 19 22:14:34 2017  clohfink:Joined the channel
Fri May 19 22:16:24 2017  clohfink:Joined the channel
Fri May 19 23:22:33 2017  dikang:Joined the channel
Fri May 19 23:34:59 2017  tolbertam:Joined the channel
Fri May 19 23:48:44 2017  zaller:Joined the channel
Fri May 19 23:52:51 2017  clohfink:Joined the channel
Fri May 19 23:53:28 2017  nickmbailey:Joined the channel

Comments