Displaying #cassandra-dev/2017-08-09.log:

Wed Aug 9 00:08:19 2017  zaller:Joined the channel
Wed Aug 9 00:19:58 2017  bbromhead:Joined the channel
Wed Aug 9 02:18:18 2017  jmckenzie_:Joined the channel
Wed Aug 9 02:26:55 2017  jmckenzie:Joined the channel
Wed Aug 9 02:46:03 2017  zaller:Joined the channel
Wed Aug 9 03:29:14 2017  clohfink:Joined the channel
Wed Aug 9 03:51:17 2017  bbromhead:Joined the channel
Wed Aug 9 04:10:11 2017  bbromhead:Joined the channel
Wed Aug 9 04:32:19 2017  Guest73:Joined the channel
Wed Aug 9 04:49:40 2017  Guest73:Joined the channel
Wed Aug 9 05:57:58 2017  ostefano:Joined the channel
Wed Aug 9 06:36:45 2017  cosql:Joined the channel
Wed Aug 9 06:48:46 2017  Guest73:Joined the channel
Wed Aug 9 06:51:13 2017  spodkowinski:Joined the channel
Wed Aug 9 07:24:23 2017  cosql:Joined the channel
Wed Aug 9 07:53:59 2017  ostefano:Joined the channel
Wed Aug 9 08:50:17 2017  RussSpitzer:Joined the channel
Wed Aug 9 09:31:10 2017  mikewallace:Joined the channel
Wed Aug 9 09:31:10 2017  urandom:Joined the channel
Wed Aug 9 09:31:23 2017  normanm:Joined the channel
Wed Aug 9 09:32:18 2017  jimInDevNull:Joined the channel
Wed Aug 9 09:32:40 2017  tjake:Joined the channel
Wed Aug 9 12:05:54 2017  RussSpitzer_:Joined the channel
Wed Aug 9 12:38:06 2017  clohfink:Joined the channel
Wed Aug 9 13:21:42 2017  RussSpitzer:Joined the channel
Wed Aug 9 13:52:07 2017  adamholmberg:Joined the channel
Wed Aug 9 14:37:02 2017  RussSpitzer_:Joined the channel
Wed Aug 9 15:28:55 2017  jfarrell:Joined the channel
Wed Aug 9 15:48:45 2017  damons:Joined the channel
Wed Aug 9 16:07:52 2017  Guest73:Joined the channel
Wed Aug 9 16:14:06 2017  jayzhuang:Joined the channel
Wed Aug 9 16:16:24 2017  jeffj:Joined the channel
Wed Aug 9 16:16:44 2017  jeffj:Joined the channel
Wed Aug 9 17:09:42 2017  bbromhead:Joined the channel
Wed Aug 9 17:58:36 2017  RussSpitzer:Joined the channel
Wed Aug 9 18:00:28 2017  adamholm_:Joined the channel
Wed Aug 9 18:20:18 2017  kvaster:Joined the channel
Wed Aug 9 18:22:15 2017  jayzhuang:Joined the channel
Wed Aug 9 18:26:10 2017  jayzhuang:Joined the channel
Wed Aug 9 18:30:38 2017  adamholmberg:Joined the channel
Wed Aug 9 18:34:32 2017  robusto:Joined the channel
Wed Aug 9 18:36:12 2017  robusto:Joined the channel
Wed Aug 9 18:59:26 2017  kvaster:Joined the channel
Wed Aug 9 19:38:18 2017  robusto_:Joined the channel
Wed Aug 9 19:41:02 2017  jayzhuang:Joined the channel
Wed Aug 9 19:43:43 2017  urandom:anyone familiar with any open bugs (affecting 2.2.6) that would cause issues creating keyspaces and tables?
Wed Aug 9 19:44:12 2017  urandom:and, issues as in, mismatching cfIds
Wed Aug 9 19:45:39 2017  driftx:usually when I see that, it's from simultaneous creation, which you shouldn't do.
Wed Aug 9 19:47:19 2017  driftx:if that's the case it's fixable, but kind of a pain
Wed Aug 9 19:47:27 2017  urandom:but that creates schema disagreement exceptions, no?
Wed Aug 9 19:47:32 2017  urandom:fixable?
Wed Aug 9 19:47:52 2017  urandom:ERROR [MigrationStage:1] 2017-08-09 19:00:59,932 CassandraDaemon.java:185 - Exception in thread Thread[MigrationStage:1,5,main]
Wed Aug 9 19:47:55 2017  urandom:org.apache.cassandra.exceptions.ConfigurationException: Column family ID mismatch (found 148dde70-7d35-11e7-b02f-55a3ee27dcc3; expected 0d663430-7d35-11e7-b60
Wed Aug 9 19:47:58 2017  urandom:2-91a5081ac860)
Wed Aug 9 19:50:51 2017  driftx:do you have different uuids in the data dirs on different machines?
Wed Aug 9 19:52:24 2017  urandom:whoa
Wed Aug 9 19:52:34 2017  urandom:there is more than one directory there
Wed Aug 9 19:52:58 2017  adamholm_:Joined the channel
Wed Aug 9 19:53:24 2017  driftx:?
Wed Aug 9 19:53:48 2017  driftx:yes, ok, so you've been the victim of simultaneous creation
Wed Aug 9 19:54:10 2017  urandom:there are multiple directories for each of the tables in question
Wed Aug 9 19:54:12 2017  urandom:well 2
Wed Aug 9 19:54:23 2017  driftx:with different uuids?
Wed Aug 9 19:54:24 2017  urandom:i wonder if they where trying to deploy this more than once
Wed Aug 9 19:54:25 2017  urandom:yes
Wed Aug 9 19:54:35 2017  driftx:yep, simultaneous creation
Wed Aug 9 19:55:01 2017  driftx:rolling restart so the chema agrees on a winner, then move the loser into the winner if needed and refresh
Wed Aug 9 19:55:53 2017  urandom:rolling restart of the entire cluster, you mean?
Wed Aug 9 19:56:11 2017  urandom:i think this was from two separate attempts, as much as an hour apr
Wed Aug 9 19:56:13 2017  urandom:apart
Wed Aug 9 19:56:28 2017  urandom:based on the ids and what i see logged
Wed Aug 9 19:57:32 2017  driftx:that is generally what rolling restart means, yes :)
Wed Aug 9 19:57:46 2017  urandom:takes hours... ugh
Wed Aug 9 19:58:08 2017  driftx:check the timestamps on the data dirs
Wed Aug 9 19:58:10 2017  urandom:#fml
Wed Aug 9 19:58:31 2017  urandom:good idea
Wed Aug 9 19:58:44 2017  urandom:oh, 4 minutes apart
Wed Aug 9 19:59:04 2017  driftx:still seems like a long time for propagation
Wed Aug 9 20:00:22 2017  urandom:72 nodes
Wed Aug 9 20:01:07 2017  urandom:but... there are two tables in this keyspace, and there are two directories for each
Wed Aug 9 20:01:22 2017  urandom:and each is 4 minutes apart from it twin
Wed Aug 9 20:01:28 2017  urandom:its
Wed Aug 9 20:03:37 2017  driftx:weird
Wed Aug 9 20:03:56 2017  driftx:maybe some kind of partition that healed
Wed Aug 9 20:10:38 2017  bbromhead:Joined the channel
Wed Aug 9 20:24:46 2017  urandom:¯\_(ツ)_/¯
Wed Aug 9 21:00:31 2017  jayzhuang:Joined the channel
Wed Aug 9 21:43:47 2017  adamholmberg:Joined the channel
Wed Aug 9 21:57:05 2017  neil_:Joined the channel
Wed Aug 9 22:12:34 2017  kohlisankalp:Joined the channel
Wed Aug 9 22:13:17 2017  kohlisan_:Joined the channel
Wed Aug 9 22:44:22 2017  jayzhuang:Joined the channel
Wed Aug 9 22:55:51 2017  kohlisankalp:Joined the channel
Wed Aug 9 22:59:07 2017  bbromhead:Joined the channel
Wed Aug 9 22:59:47 2017  bbromhead:Joined the channel
Wed Aug 9 23:00:03 2017  jayzhuang:Joined the channel
Wed Aug 9 23:02:18 2017  kohlisankalp:Joined the channel
Wed Aug 9 23:03:33 2017  jayzhuang:Joined the channel

Comments