Displaying #cassandra-dev/2017-03-15.log:

Wed Mar 15 00:07:27 2017  tolbertam:Joined the channel
Wed Mar 15 00:19:38 2017  tolbertam:Joined the channel
Wed Mar 15 00:46:21 2017  tolbertam:Joined the channel
Wed Mar 15 01:32:21 2017  kohlisankalp:Joined the channel
Wed Mar 15 02:36:08 2017  alourie:@ptnapoleon I’d love to get there. Any pointers on how would I resolve errors? For example, I have auth_test.py:TestAuth.handle_corrupt_role_data_test always failing with NPE in the cluster. Tried my mac, my home computer and my jenkins host, all fail with the same error.
Wed Mar 15 03:14:34 2017  dilzio:Joined the channel
Wed Mar 15 03:19:38 2017  alourie:Joined the channel
Wed Mar 15 03:20:18 2017  dikang:Joined the channel
Wed Mar 15 03:28:44 2017  Vijay:Joined the channel
Wed Mar 15 04:05:22 2017  kohlisankalp:Joined the channel
Wed Mar 15 04:20:27 2017  alourie:is cassci a part of the Apache now? If that’s the case, is it possible to get an access to the testing setup scripts/code? We’d like to replicate it as closely as possible and possibly improve.
Wed Mar 15 04:52:43 2017  jeffj:cassci is not part of the apache project, and cassci is being deprecated. the replacement is an open question (certainly asf jenkins is likely, but limited)
Wed Mar 15 04:53:08 2017  jeffj:and i missed him :(
Wed Mar 15 05:09:48 2017  dilzio:Joined the channel
Wed Mar 15 05:20:31 2017  alourie:Joined the channel
Wed Mar 15 06:14:21 2017  alourie:Joined the channel
Wed Mar 15 07:01:41 2017  cosql:Joined the channel
Wed Mar 15 07:14:08 2017  kvaster:Joined the channel
Wed Mar 15 07:36:38 2017  kvaster:Joined the channel
Wed Mar 15 07:52:00 2017  spodkowinski:Joined the channel
Wed Mar 15 08:17:24 2017  cosql:Joined the channel
Wed Mar 15 08:18:36 2017  cosql:Joined the channel
Wed Mar 15 08:26:52 2017  kvaster:Joined the channel
Wed Mar 15 08:53:17 2017  alourie:Joined the channel
Wed Mar 15 08:57:18 2017  kvaster_:Joined the channel
Wed Mar 15 09:01:43 2017  kvaster:Joined the channel
Wed Mar 15 10:06:26 2017  cosql:Joined the channel
Wed Mar 15 10:08:03 2017  cosql:Joined the channel
Wed Mar 15 10:09:23 2017  cosql:Joined the channel
Wed Mar 15 10:10:45 2017  cosql:Joined the channel
Wed Mar 15 10:11:51 2017  cosql:Joined the channel
Wed Mar 15 10:12:56 2017  cosql:Joined the channel
Wed Mar 15 10:14:07 2017  cosql:Joined the channel
Wed Mar 15 10:15:58 2017  cosql:Joined the channel
Wed Mar 15 10:16:31 2017  cosql:Joined the channel
Wed Mar 15 11:39:09 2017  alourie:@jeffj thanks for cassci info. a bit unfortunate, but ok.
Wed Mar 15 12:01:40 2017  adamholmberg:Joined the channel
Wed Mar 15 12:03:54 2017  adamholmberg:Joined the channel
Wed Mar 15 12:25:17 2017  makrusak:Joined the channel
Wed Mar 15 13:05:30 2017  adamholmberg:Joined the channel
Wed Mar 15 13:27:30 2017  Gassston:Joined the channel
Wed Mar 15 13:29:48 2017  Gassston:Hi! I know this is not the right place to ask, but I' cant get help from others forums :). I'm running 1.2 version and accidentally deleted some data. Is there a way to restore this data to the previos version (before the delete)? I know the old data is still there, but cant' figure out how to revover it.
Wed Mar 15 13:41:15 2017  juanV:Joined the channel
Wed Mar 15 14:16:24 2017  ptnapoleon:alourie: If youre still here, it mostly is possible to get access to the testing scripts. At the very least, the scripts that were moved from cassci to the asf jenkins are all open
Wed Mar 15 14:17:17 2017  alourie:@ptnapoleon ok, so how would I get the access to that? I’d be quite interested in job runners that split tests into multiple slave nodes
Wed Mar 15 14:19:02 2017  ptnapoleon:I don't believe we actually open sourced that bit, but it's fairly trivial to implement yourself. The dtests use the nosetests framework, which can generate a list of tests, and also run a [sub-]list of tests, so it's a simple matter of splitting up that initial list, and scp-ing the subsets to the slave nodes
Wed Mar 15 14:20:12 2017  alourie:while that’s true, I think that using someone’s already existing code would save much time :-)
Wed Mar 15 14:20:32 2017  ptnapoleon:Not when you account how long it would take me to get approval to open source it
Wed Mar 15 14:20:45 2017  alourie:ha
Wed Mar 15 14:20:56 2017  alourie:I guess that’ll take forecer
Wed Mar 15 14:21:00 2017  alourie:forever*
Wed Mar 15 14:22:11 2017  ptnapoleon:I can look into it. I'd guess a month or so, with no guarantees it actually occurs, obviously?
Wed Mar 15 14:22:40 2017  tolbertam:Joined the channel
Wed Mar 15 14:24:37 2017  alourie:Sure. Would love to have a look at that code. I bet CI that would run with Apache Project could benefit from that too.
Wed Mar 15 14:25:29 2017  ptnapoleon:There aren't enough slave nodes on the ASF CI for C* to benefit
Wed Mar 15 14:25:38 2017  ptnapoleon:But, to help people setting something up internal, we'd maybe do it
Wed Mar 15 14:25:41 2017  ptnapoleon:I'll ask tomorrow
Wed Mar 15 14:26:03 2017  alourie:Indeed, we want this for internal use. Thanks, much appreciated.
Wed Mar 15 14:37:55 2017  thobbs:Joined the channel
Wed Mar 15 14:40:55 2017  kvaster:Joined the channel
Wed Mar 15 14:42:17 2017  kvaster:Joined the channel
Wed Mar 15 14:52:24 2017  tolbertam:Joined the channel
Wed Mar 15 15:03:23 2017  cosql:Joined the channel
Wed Mar 15 15:04:32 2017  cosql:Joined the channel
Wed Mar 15 15:06:01 2017  cosql:Joined the channel
Wed Mar 15 15:07:31 2017  cosql:Joined the channel
Wed Mar 15 15:08:44 2017  cosql:Joined the channel
Wed Mar 15 15:21:14 2017  joaquin:Joined the channel
Wed Mar 15 15:39:12 2017  jasobrown:ptnapoleon: i would appreciate anything you would be able to share, as well
Wed Mar 15 15:39:21 2017  jasobrown:any notes or hints or tips ;)
Wed Mar 15 15:39:55 2017  ptnapoleon:jasobrown: So I know exlt has already released some stuff, but he's on vacation, and I don't actually know where those scripts are. We'll be sure to make you aware of everything
Wed Mar 15 15:45:06 2017  pcmanus:Joined the channel
Wed Mar 15 15:46:00 2017  spodkowinski:https://github.com/apache/cassandra-builds ?
Wed Mar 15 15:46:25 2017  ptnapoleon:That's definitely a lot of it
Wed Mar 15 16:01:15 2017  mpenick_:Joined the channel
Wed Mar 15 16:03:59 2017  kohlisankalp:Joined the channel
Wed Mar 15 16:11:47 2017  Vijay:Joined the channel
Wed Mar 15 16:52:20 2017  tolbertam:Joined the channel
Wed Mar 15 17:03:30 2017  dikang:Joined the channel
Wed Mar 15 18:06:15 2017  dikang:Joined the channel
Wed Mar 15 18:11:04 2017  nowhammy:Joined the channel
Wed Mar 15 18:12:16 2017  nowhammy:Joined the channel
Wed Mar 15 18:26:17 2017  kvaster:Joined the channel
Wed Mar 15 18:26:53 2017  Vijay:Joined the channel
Wed Mar 15 19:03:57 2017  Vijay:Joined the channel
Wed Mar 15 19:31:27 2017  kohlisankalp:Joined the channel
Wed Mar 15 20:13:20 2017  EdwardCapriolo:Joined the channel
Wed Mar 15 20:37:00 2017  aweisberg:Can I get a CAS expert to tell me if what I am trying to do in https://issues.apache.org/jira/browse/CASSANDRA-13327 "Pending endpoints size check for CAS doesn't play nicely with writes-on-replacement" is sane?
Wed Mar 15 20:45:24 2017  thobbs:your best bet is probably to request feedback from Sylvain on the ticket
Wed Mar 15 20:45:32 2017  thobbs:or bdeggleston
Wed Mar 15 20:53:12 2017  EdwardCapriolo:aweisberg: I know this topic may have been discussed before and don't get me wrong I really dislike mockito, but should cassandra use mockito so things like this can have an explicit test?
Wed Mar 15 20:54:44 2017  EdwardCapriolo:Inline comments like //Host replacements shouldn't count as a pending range movement since it's not a movement. are a strong indicator that preconditions exists and pendingEndpoints.size() seems easily mockable
Wed Mar 15 20:55:23 2017  aweisberg:That just puts together the kind of unit test that doesn't actually test anything. For instance in a real cluster started the real way what will the contents of TMD be?
Wed Mar 15 20:55:46 2017  aweisberg:I might be more confident in a unit test if there were some kind of clear contract and documentation there.
Wed Mar 15 20:56:11 2017  EdwardCapriolo:Well your essentially adminting since you cant reall test it that no one takes it out without understanding it
Wed Mar 15 20:56:33 2017  aweisberg:I more or less don't know the true definition of pending range even. I know what it looks like implicitly based on the implementation. But architecturally?
Wed Mar 15 20:57:06 2017  aweisberg:EdwardCapriolo: When did I say I couldn't test it?
Wed Mar 15 20:57:18 2017  EdwardCapriolo:Im saying i dont see a specific test in your patch.
Wed Mar 15 20:57:29 2017  aweisberg:EdwardCapriolo: Comment presenting the branch says there is a dtest in the works
Wed Mar 15 20:57:54 2017  aweisberg:but that won't tell if you it's correct it will only tell you it behaves as intended. It might still cause stale reads.
Wed Mar 15 20:57:59 2017  cosql:Joined the channel
Wed Mar 15 20:59:35 2017  EdwardCapriolo:sure. Well your patch implies that the person who added the cas feature did not account for pending range movements, your patch may fix it, but your counting on a dtest that can do the action inside a pending range move. Unless you have a lot of data I do not follow how a dtest can be sure of that. Moving a few thousand rows happens very quickly right?
Wed Mar 15 21:00:05 2017  aweisberg:EdwardCapriolo: Move via what? Are you talking about streaming? You just block streaming with byteman and have it stall indefinitely.
Wed Mar 15 21:00:58 2017  EdwardCapriolo:I am suggesting that if pendingEndpoints.size() is non 0 the cluster is changing somehow.
Wed Mar 15 21:01:56 2017  EdwardCapriolo:Even if you were to black box test by doing 1000 CAS during a move/join you are not actually sure your code path got hit are you?
Wed Mar 15 21:03:12 2017  aweisberg:The request will return unavailable if it doesn't hit that line. So pretty damn sure. And if you want to review I'm happy to instrument it with byteman to know it actually went there.
Wed Mar 15 21:03:50 2017  aweisberg:So do you always check for code coverage manually in unit tests that you write?
Wed Mar 15 21:03:56 2017  aweisberg:Or do you test behavior?
Wed Mar 15 21:04:35 2017  EdwardCapriolo:Well im not a person that does manual QA so I value code coverage over testing
Wed Mar 15 21:05:08 2017  aweisberg:On that we agree. Hence the dtest
Wed Mar 15 21:05:49 2017  aweisberg:So what is your definition of code coverage? Tests behavior or looks to make sure a line ran via injecting a coverage detection?
Wed Mar 15 21:05:56 2017  EdwardCapriolo:I dont think you need a dtest to confirm a branch was hit. If pending >0 assert this happened if pending =0 assert that happened
Wed Mar 15 21:06:35 2017  kohlisankalp:Joined the channel
Wed Mar 15 21:08:51 2017  EdwardCapriolo:This is not a foreign idea. I notice there are already classes in cassandra test code named MockXyz.
Wed Mar 15 21:09:18 2017  aweisberg:OK so it isn't about coverage. It's about testing behavior.
Wed Mar 15 21:09:20 2017  EdwardCapriolo:Could just use mockito/verify and say the same thing without making a class named MockXyz
Wed Mar 15 21:09:59 2017  aweisberg:OK, say you mock, how do you construct a working multi-process representation of how C* actually works from within a unit test without running the code that actually constructs the cluster?
Wed Mar 15 21:10:13 2017  aweisberg:How do you know the TMD you have is the TMD real people have?
Wed Mar 15 21:10:30 2017  aweisberg:How do you know that it doesn't throw unavailable in the future for some other reason?
Wed Mar 15 21:10:53 2017  aweisberg:Unit tests and mocks have their place, but they are not a universal answer.
Wed Mar 15 21:11:05 2017  EdwardCapriolo:How much code would it be to add a mock test 2 lines?
Wed Mar 15 21:11:21 2017  aweisberg:It doesn't actually test it is the point.
Wed Mar 15 21:11:46 2017  aweisberg:That demonstrates those two lines have the intended effect. It doesn't say anything about the behavior of the system as a whole. Which incidentally is what the ticket is about.
Wed Mar 15 21:11:49 2017  EdwardCapriolo:when someone does not understand/consider this case and removes/breaks your assertion
Wed Mar 15 21:11:55 2017  aweisberg:The user doesn't care your two lines do something. They care that the feature works.
Wed Mar 15 21:12:02 2017  EdwardCapriolo:another ticket opens to fix it again
Wed Mar 15 21:12:08 2017  aweisberg:No a test fails
Wed Mar 15 21:12:33 2017  aweisberg:Which yeah we ignore, but that is an orthogonal issue.
Wed Mar 15 21:12:54 2017  EdwardCapriolo:A test that takes about 19 times the computation to execute :)
Wed Mar 15 21:14:19 2017  EdwardCapriolo:Its a simple suggestion and an option. Preconditions written in comments almost translate directly to mockable tests. Your free to disagree.
Wed Mar 15 21:14:33 2017  aweisberg:I don't disagree that it could be unit tested. It's just not good enough.
Wed Mar 15 21:14:43 2017  EdwardCapriolo:Go write a d test that takes 6 minutes to run to solve the same thing
Wed Mar 15 21:15:03 2017  aweisberg:It's not the same. And you have yet to say anything to contradict that. You just repeat yourself.
Wed Mar 15 21:16:41 2017  EdwardCapriolo:For someone asking for help you seem to have a lot of answers. Why I dont deal with c* much
Wed Mar 15 21:16:47 2017  EdwardCapriolo:Left the channel
Wed Mar 15 21:18:37 2017  Bhaskar:Joined the channel
Wed Mar 15 21:19:41 2017  MarcintheCloud:Joined the channel
Wed Mar 15 21:20:45 2017  sequoyha:Joined the channel
Wed Mar 15 21:39:06 2017  dikang:Joined the channel
Wed Mar 15 21:45:32 2017  EdwardCapriolo:Joined the channel
Wed Mar 15 21:45:56 2017  EdwardCapriolo:Joined the channel
Wed Mar 15 21:46:02 2017  EdwardCapriolo:Left the channel
Wed Mar 15 22:22:01 2017  tolbertam:Joined the channel
Wed Mar 15 22:36:50 2017  Vijay:Joined the channel
Wed Mar 15 23:55:43 2017  Bhaskar:Joined the channel

Comments