Displaying #cassandra-dev/2016-09-14.log:

Wed Sep 14 00:09:53 2016  kohlisankalp:Joined the channel
Wed Sep 14 00:12:15 2016  ArunThekkiniyath:Joined the channel
Wed Sep 14 00:16:11 2016  clohfink:Joined the channel
Wed Sep 14 00:24:18 2016  kohlisankalp:Joined the channel
Wed Sep 14 00:37:48 2016  kohlisankalp:Joined the channel
Wed Sep 14 00:49:45 2016  kohlisankalp:Joined the channel
Wed Sep 14 00:54:46 2016  kohlisankalp:Joined the channel
Wed Sep 14 01:01:48 2016  kohlisankalp:Joined the channel
Wed Sep 14 01:22:54 2016  mstepura:Joined the channel
Wed Sep 14 01:23:36 2016  kohlisankalp:Joined the channel
Wed Sep 14 01:32:57 2016  clohfink:Joined the channel
Wed Sep 14 01:33:14 2016  clohfink:Joined the channel
Wed Sep 14 01:35:10 2016  clohfink:Joined the channel
Wed Sep 14 01:45:23 2016  clohfink_:Joined the channel
Wed Sep 14 02:03:12 2016  kohlisankalp:Joined the channel
Wed Sep 14 02:21:50 2016  tolbertam:Joined the channel
Wed Sep 14 02:29:01 2016  mstepura:Joined the channel
Wed Sep 14 02:41:56 2016  mstepura:Joined the channel
Wed Sep 14 02:44:43 2016  tolbertam:Joined the channel
Wed Sep 14 02:47:36 2016  mstepura:Joined the channel
Wed Sep 14 02:53:32 2016  mstepura:Joined the channel
Wed Sep 14 02:54:22 2016  mstepura:Joined the channel
Wed Sep 14 02:54:26 2016  jfarrell:ping driftx exlt
Wed Sep 14 02:54:39 2016  driftx:sup
Wed Sep 14 02:55:00 2016  driftx:all I know is he tried his committer creds, they didn't work, and I was out of ideas
Wed Sep 14 02:55:07 2016  mstepura:Joined the channel
Wed Sep 14 02:55:18 2016  jfarrell:dist.a.o is for PMC to push to, we do not have committers_may_release set
Wed Sep 14 02:55:29 2016  driftx:ah, I see.
Wed Sep 14 02:55:57 2016  mstepura:Joined the channel
Wed Sep 14 02:56:42 2016  mstepura:Joined the channel
Wed Sep 14 02:57:18 2016  jfarrell:goes to find the script …
Wed Sep 14 02:57:32 2016  mstepura:Joined the channel
Wed Sep 14 02:57:51 2016  driftx:wait, we can fix it?
Wed Sep 14 02:58:09 2016  driftx:I took that as "only pmc can do it"
Wed Sep 14 02:58:22 2016  mstepura:Joined the channel
Wed Sep 14 02:59:04 2016  jfarrell:correct, unless we want committers to be able to release
Wed Sep 14 02:59:07 2016  mstepura:Joined the channel
Wed Sep 14 02:59:09 2016  jfarrell:see - https://git1-us-west.apache.org/repos/asf?p=infrastructure-puppet.git;a=blob;f=modules/subversion_server/files/authorization/generate-dist-authorization;h=defda37a14a181a9db70d01c3e7f37d535b047d8;hb=HEAD#l14
Wed Sep 14 02:59:45 2016  driftx:I think we're ok with that
Wed Sep 14 02:59:56 2016  driftx:alternatively, I guess we could make him pmc, heh
Wed Sep 14 03:00:38 2016  driftx:either way sounds like a vote is required?
Wed Sep 14 03:01:51 2016  jfarrell:well start private@ discussion for either, but in any case nothing to do right now and will comment on 12239
Wed Sep 14 03:02:01 2016  driftx:or I guess someone on pmc can just add the key?
Wed Sep 14 03:02:09 2016  driftx:ok, thanks
Wed Sep 14 03:02:25 2016  jfarrell:pmc* could add the key and would also have to push the artifacts
Wed Sep 14 03:03:24 2016  driftx:hmm, sounds suboptimal
Wed Sep 14 03:03:53 2016  mstepura:Joined the channel
Wed Sep 14 03:04:22 2016  jfarrell:a pmc member will have to do the release
Wed Sep 14 03:04:50 2016  driftx:even with committers_may_release?
Wed Sep 14 03:05:15 2016  jfarrell:then they could
Wed Sep 14 03:05:29 2016  driftx:k
Wed Sep 14 03:06:59 2016  jfarrell:72h vote, 1m puppet change, wait for puppet to run or find nearest root@ to kick puppet on dist.a.o
Wed Sep 14 03:29:31 2016  dikang:Joined the channel
Wed Sep 14 03:34:51 2016  dikang:Joined the channel
Wed Sep 14 03:35:38 2016  JayZhuang:Joined the channel
Wed Sep 14 03:57:53 2016  mstepura:Joined the channel
Wed Sep 14 05:25:02 2016  kohlisankalp:Joined the channel
Wed Sep 14 05:29:09 2016  dikang:Joined the channel
Wed Sep 14 06:13:11 2016  mstepura:Joined the channel
Wed Sep 14 06:40:28 2016  mstepura:Joined the channel
Wed Sep 14 06:41:18 2016  mstepura:Joined the channel
Wed Sep 14 06:42:03 2016  mstepura:Joined the channel
Wed Sep 14 06:42:50 2016  mstepura:Joined the channel
Wed Sep 14 06:43:40 2016  mstepura:Joined the channel
Wed Sep 14 06:44:30 2016  mstepura:Joined the channel
Wed Sep 14 06:45:22 2016  mstepura:Joined the channel
Wed Sep 14 06:46:04 2016  mstepura:Joined the channel
Wed Sep 14 06:46:51 2016  mstepura:Joined the channel
Wed Sep 14 06:47:42 2016  mstepura:Joined the channel
Wed Sep 14 06:48:27 2016  mstepura:Joined the channel
Wed Sep 14 07:16:23 2016  dikang:Joined the channel
Wed Sep 14 07:43:03 2016  joseph_arnstein:Joined the channel
Wed Sep 14 08:12:40 2016  minimarcel:Joined the channel
Wed Sep 14 10:25:09 2016  ArunThekkiniyath:Joined the channel
Wed Sep 14 10:51:04 2016  spodkowinski:Joined the channel
Wed Sep 14 12:18:42 2016  Mitch__:Joined the channel
Wed Sep 14 12:23:43 2016  minimarcel:Joined the channel
Wed Sep 14 13:43:11 2016  sam__:Joined the channel
Wed Sep 14 13:45:21 2016  sam__:hi guys
Wed Sep 14 13:45:47 2016  sam__:Good UGT
Wed Sep 14 13:46:30 2016  sam__:i need an urgent help from u guys, can anyone help me ?
Wed Sep 14 13:49:25 2016  exlt:thanks for the help on dist.a.o, jfarrell
Wed Sep 14 13:49:25 2016  sam__:i was struggling in cassandra migration from 2.1 to 3.7
Wed Sep 14 13:50:17 2016  sam__:hi exlt
Wed Sep 14 13:50:29 2016  exlt:driftx: so there is a discussion on private@? I can send a "should we add cassandra to committers_may_release?" to dev@, if needed
Wed Sep 14 13:51:40 2016  exlt:sam__: have you asked for help on #cassandra? did you read NEWS.txt, etc.?
Wed Sep 14 13:52:06 2016  exlt:this channel is about the development of cassandra (see topic)
Wed Sep 14 13:53:25 2016  sam__:i'm new to this channel, exlt
Wed Sep 14 13:53:46 2016  sam__:could u tell whats that NEWS.txt ?
Wed Sep 14 13:54:15 2016  exlt:upgrade notes - again, another really good question for #cassandra ;)
Wed Sep 14 13:57:45 2016  sam__:exlt: i'm getting an data related issue, so i thought it can be addressed easily by devs
Wed Sep 14 13:58:17 2016  jfarrell:morning exlt, some more general disucssions occurring on private@, for now to keep things unblocked a pmc member would have to do the rc
Wed Sep 14 13:59:03 2016  exlt:OK, thanks
Wed Sep 14 14:00:07 2016  sam__:exlt, getting this error - https://gist.github.com/anonymous/1132cc3d9aa038ae32b4ed24a14178ce
Wed Sep 14 14:00:32 2016  exlt:sam__: Channel guidelines: http://bit.ly/bCbK2R
Wed Sep 14 14:00:42 2016  exlt:ask on #cassandra, please
Wed Sep 14 14:03:51 2016  sam__:exlt, thanks for that. will try there...
Wed Sep 14 14:10:15 2016  adamholmberg:Joined the channel
Wed Sep 14 14:13:46 2016  devdazed:Joined the channel
Wed Sep 14 14:50:28 2016  gdusbabek:someone please help me out: what is mshulers IRC nick?
Wed Sep 14 14:50:42 2016  beobal:exlt
Wed Sep 14 14:50:47 2016  gdusbabek:thx.
Wed Sep 14 14:53:54 2016  adamholmberg:Joined the channel
Wed Sep 14 15:01:45 2016  tolbertam:Joined the channel
Wed Sep 14 15:32:59 2016  thobbs:Joined the channel
Wed Sep 14 15:45:49 2016  adamholmberg:Joined the channel
Wed Sep 14 15:55:50 2016  minimarcel:Joined the channel
Wed Sep 14 15:57:59 2016  maedhroz:Joined the channel
Wed Sep 14 16:02:12 2016  minimarcel:Joined the channel
Wed Sep 14 16:19:24 2016  mstepura:Joined the channel
Wed Sep 14 16:19:24 2016  clohfink:Joined the channel
Wed Sep 14 16:23:47 2016  JayZhuang:Joined the channel
Wed Sep 14 16:44:25 2016  dikang:Joined the channel
Wed Sep 14 17:04:55 2016  minimarcel:Joined the channel
Wed Sep 14 17:10:51 2016  dikang:Joined the channel
Wed Sep 14 17:13:40 2016  clohfink:Joined the channel
Wed Sep 14 17:34:28 2016  dikang:Joined the channel
Wed Sep 14 17:43:34 2016  mkjellman:Joined the channel
Wed Sep 14 17:43:40 2016  mkjellman:who wants to talk stress with me? :)
Wed Sep 14 17:43:49 2016  mkjellman:i'm getting stressed because of stress.
Wed Sep 14 17:49:53 2016  adamholmberg:Joined the channel
Wed Sep 14 17:50:30 2016  clohfink:Joined the channel
Wed Sep 14 18:19:12 2016  jkni:I might be able to help - what's up?
Wed Sep 14 18:19:47 2016  mkjellman:i haven't been able to get a stable config to test actually wide partitions despite multiple days of trying to figure this out
Wed Sep 14 18:20:03 2016  mkjellman:stress generally gc's itself to death generating the batches
Wed Sep 14 18:20:38 2016  mkjellman:the configs i looked at from what jake had sent me over to try ended up creating very small rows
Wed Sep 14 18:21:01 2016  mkjellman:ideally would like to get a read/write mixed load with a 10-100GB target size per row
Wed Sep 14 18:21:08 2016  kohlisankalp:Joined the channel
Wed Sep 14 18:21:25 2016  mkjellman:i'm pretty close to just writing something myself
Wed Sep 14 18:21:35 2016  mkjellman:no one seems to know how stress works, it's so un-obvious how things are related
Wed Sep 14 18:21:58 2016  mkjellman:and it definitely doesn't seem to have been written with testing of even medium sized partitions
Wed Sep 14 18:27:07 2016  adamholmberg:Joined the channel
Wed Sep 14 18:29:59 2016  zznate:chbatey ^
Wed Sep 14 18:30:22 2016  mkjellman:i'm 10 minutes away from writing something myself
Wed Sep 14 18:30:31 2016  mkjellman:i've been pounding my head against this for 5 days now
Wed Sep 14 18:31:17 2016  jkni:I'm afraid you've already gone farther on that particular issue than I ever have, then. Sorry to not be of help
Wed Sep 14 18:31:41 2016  mkjellman:i mean, how are we actually testing stuff if no one knows how to put load on our code..
Wed Sep 14 18:33:44 2016  zznate:mkjellman - chbatey is/has been doing a bunch of patches on stress, i know that was on his list, but not sure how far he got
Wed Sep 14 18:34:15 2016  kohlisankalp:Joined the channel
Wed Sep 14 18:34:51 2016  mkjellman:well, i think i'm gonna give up on stress nate... it would be great if someone can guide me and the community in the right direction
Wed Sep 14 18:35:23 2016  kohlisankalp:Joined the channel
Wed Sep 14 18:35:31 2016  mkjellman:we should have a easy way to just put infinite load on a cluster with like small/medium/large row configs and a mixed read/write workload
Wed Sep 14 18:35:35 2016  mkjellman:nothing fancy..
Wed Sep 14 18:39:23 2016  zznate:agreed. honestly, i find stress very difficult to use and have to go back and look at my notes/previous work everytime I need to use it.
Wed Sep 14 18:42:20 2016  thobbs:pretty much everybody does
Wed Sep 14 18:42:30 2016  thobbs:we need a v3 stress
Wed Sep 14 18:43:00 2016  mkjellman:i wrote a iterator implementation that generates timeuuids for the birch unit tests and samples the output at a given rate
Wed Sep 14 18:43:27 2016  mkjellman:that way i can generate unlimited sorted ordered data in an efficient way and still go back and do reads to validate the inserts
Wed Sep 14 18:43:58 2016  mstepura:Joined the channel
Wed Sep 14 18:44:06 2016  mkjellman:i think i'm just gonna throw together something that creates n client threads and with n keys per thread and uses timeuuids to grow the rows
Wed Sep 14 18:44:24 2016  tjake:Wide partitions are not something stress can do, mainly since it needs to track the order of what's been written
Wed Sep 14 18:44:28 2016  mkjellman:all of the metric stuff that stress collects today is unnecessary imho... i just need something to drive read and write load
Wed Sep 14 18:44:30 2016  tjake:yeah I would agree
Wed Sep 14 18:44:42 2016  tjake:this is why I ended up writing mvbench
Wed Sep 14 18:44:51 2016  mkjellman:oh you already wrote something jake?!
Wed Sep 14 18:44:56 2016  mkjellman:why have you been holding out on me..
Wed Sep 14 18:45:06 2016  tjake:no I mean to stress MVs
Wed Sep 14 18:45:17 2016  tjake:not wide rows, although it can maybe
Wed Sep 14 18:45:31 2016  minimarcel:Joined the channel
Wed Sep 14 18:45:37 2016  tjake:https://github.com/tjake/mvbench
Wed Sep 14 18:45:39 2016  mkjellman:the problem today is this generate everything up front thing doesn't work
Wed Sep 14 18:45:58 2016  dikang:Joined the channel
Wed Sep 14 18:46:27 2016  tjake:Yeah, I think we need better control over specific types vs just "distribution"
Wed Sep 14 18:46:42 2016  tjake:like for timestamps just have "increment by 1" mode
Wed Sep 14 18:47:11 2016  mkjellman:yeah, i mean you can just generate various giberish values and sample them at some rate
Wed Sep 14 18:47:19 2016  mkjellman:i learned this lesson by using these massive word lists originally
Wed Sep 14 18:47:35 2016  mkjellman:but the gc pressure that happens from just trying to keep the word lists around so you can validate order and stuff is massive
Wed Sep 14 18:47:54 2016  mkjellman:reading the code now jake... seems sane already.
Wed Sep 14 18:48:30 2016  tjake:the initial new stress version just used a mersenne twister to randomly generate data like monte carlo
Wed Sep 14 18:48:39 2016  kohlisankalp:Joined the channel
Wed Sep 14 18:48:48 2016  driftx:I'll never forget from the survey a while back: "What would you like to see in C*?" "A stress tool that doesn't require being a mathematician to use."
Wed Sep 14 18:48:49 2016  mkjellman:what was wrong with that?
Wed Sep 14 18:49:12 2016  tjake:the distribution wasn't controllable
Wed Sep 14 18:49:25 2016  mkjellman:driftx -- it's impossible! i mean the way the various column distributions play off unlogged vs logged batches of sizes.. i don't understand how it decides how many partitions are gonna get generated
Wed Sep 14 18:49:58 2016  tjake:the default can be made to "always insert 1 row at a time"
Wed Sep 14 18:50:13 2016  tjake:so the whole insert section goes away
Wed Sep 14 18:50:28 2016  mkjellman:like with fixed(1) yeah?
Wed Sep 14 18:50:38 2016  mkjellman:but if i do that i can't get any throughput at all
Wed Sep 14 18:50:53 2016  mkjellman:max i've been able to get is like 2.3k/sec on writes
Wed Sep 14 18:51:30 2016  tjake:thats what -rate threads=300 is for
Wed Sep 14 18:52:03 2016  tjake:the problem gets even harder when you have to share the state of where am I in the sequence across N threads
Wed Sep 14 18:52:07 2016  mkjellman:earlier i went -Xmx30720m -Xms30720m -XX:+UseG1GC -XX:MaxGCPauseMillis=200
Wed Sep 14 18:52:09 2016  tjake:its not a simple problem
Wed Sep 14 18:52:24 2016  tjake:if you use contents SORTED yeah
Wed Sep 14 18:52:25 2016  dikang:Joined the channel
Wed Sep 14 18:52:27 2016  tjake:it will blow up
Wed Sep 14 18:53:11 2016  mkjellman:while true; do tools/bin/cassandra-stress user profile=tools/kjtest1.yaml ops\(insert=100,simple1=100\) n=1B -rate threads=50 -pop seq=1..100 read-lookback=uniform\(1..100\) -insert visits=fixed\(100K\) revisit=uniform\(1..100K\) -node mr90p01if-ztfj06011801.mr.if.apple.com,mr90p01if-ztfj06032201.mr.if.apple.com,mr90p01if-ztfj06070501.mr.if.apple.com; sleep 2; done
Wed Sep 14 18:53:42 2016  mkjellman:kjtest1.yml => http://pastebin.com/3crMjUGy
Wed Sep 14 18:54:57 2016  tjake:why do you need to read?
Wed Sep 14 18:55:05 2016  mkjellman:because i need to validate what i put in comes back out
Wed Sep 14 18:55:10 2016  tjake:the read-lookback stuff will blow up memory
Wed Sep 14 18:55:12 2016  mkjellman:i am testing a new index after all :)
Wed Sep 14 18:55:16 2016  tjake:Well write first
Wed Sep 14 18:55:18 2016  tjake:then read
Wed Sep 14 18:55:40 2016  mkjellman:how do i configure that vs what i did
Wed Sep 14 18:55:53 2016  mkjellman:i mean it needs to know what to assert on based on what was written in, no?
Wed Sep 14 18:56:08 2016  mstepura:Joined the channel
Wed Sep 14 18:57:01 2016  tjake:yeah thats what it does
Wed Sep 14 18:57:08 2016  tjake:if you write with n=1B
Wed Sep 14 18:57:16 2016  tjake:you can validate with n=1B
Wed Sep 14 18:57:18 2016  mkjellman:that's what read-lookback does, right?
Wed Sep 14 18:57:35 2016  tjake:write lookback buffers what was acked by writers
Wed Sep 14 18:57:41 2016  tjake:but you can break it into steps
Wed Sep 14 18:58:02 2016  mkjellman:GIVE ME THE MAGIC!!!
Wed Sep 14 18:58:57 2016  tjake:tools/bin/cassandra-stress user profile=tools/kjtest1.yaml ops\(insert=100\) n=1B -rate threads=50 -pop seq=1..100
Wed Sep 14 18:59:13 2016  tjake:tools/bin/cassandra-stress user profile=tools/kjtest1.yaml ops\(simple1=100\) n=1B -rate threads=50 -pop seq=1..100
Wed Sep 14 18:59:21 2016  tjake:s/100/1/
Wed Sep 14 18:59:45 2016  mkjellman:but how does the second run know what the first run with the insert ops did?
Wed Sep 14 19:00:09 2016  tjake:because all profiles share the same seed used to seed the random generator
Wed Sep 14 19:00:44 2016  tjake:its random from a start point but always the same random
Wed Sep 14 19:01:03 2016  tjake:maths!
Wed Sep 14 19:01:06 2016  mkjellman:which 100 should be 1?
Wed Sep 14 19:01:11 2016  tjake:ops
Wed Sep 14 19:01:15 2016  tjake:doesn't matter
Wed Sep 14 19:01:34 2016  tjake:seq should probably be more than 100 though
Wed Sep 14 19:01:45 2016  tjake:you'll just update the same 100 partitions
Wed Sep 14 19:01:50 2016  tjake:unless thats what you want
Wed Sep 14 19:01:56 2016  mkjellman:i want to grow the same 100 partitions
Wed Sep 14 19:01:59 2016  tjake:ok
Wed Sep 14 19:02:20 2016  mkjellman:trying to get 10-100GB rows
Wed Sep 14 19:02:23 2016  tjake:I would restrict the partition key dist to fixed(100)
Wed Sep 14 19:02:38 2016  tjake:and use no seq
Wed Sep 14 19:02:57 2016  tjake:and make the clustering distribution fixed(1B)
Wed Sep 14 19:04:09 2016  mkjellman:while true; do tools/bin/cassandra-stress user profile=tools/kjtest1.yaml ops\(insert=100\) n=1B -rate threads=50 -node mr90p01if-ztfj06011801.mr.if.apple.com,mr90p01if-ztfj06032201.mr.if.apple.com,mr90p01if-ztfj06070501.mr.if.apple.com; tools/bin/cassandra-stress user profile=tools/kjtest1.yaml ops\(simple1=100\) n=1B -rate threads=50 -node mr90p01if-ztfj06011801.mr.if.apple.com,mr90p01if-ztfj06032201.mr.if.apple.com,mr90p01if-ztfj06070501.mr.if
Wed Sep 14 19:04:09 2016  mkjellman:.apple.com; sleep 2; done
Wed Sep 14 19:04:38 2016  tjake:why while true?
Wed Sep 14 19:04:54 2016  mkjellman:because i want to run load forever..
Wed Sep 14 19:04:56 2016  tjake:that will just update the same data over and over
Wed Sep 14 19:05:02 2016  tjake:k
Wed Sep 14 19:05:17 2016  mkjellman:is there a better way to make it go forever?
Wed Sep 14 19:05:35 2016  tjake:is 1B not enough?
Wed Sep 14 19:05:44 2016  tjake:if you get 2k/sec thats a long time :)
Wed Sep 14 19:06:21 2016  mkjellman:sure, i'll run it out of the loop and see how long it goes.
Wed Sep 14 19:06:41 2016  mkjellman:"I would restrict the partition key dist to fixed(100)"
Wed Sep 14 19:06:45 2016  tjake:if you have a beefy node for stress run with thread=800
Wed Sep 14 19:06:47 2016  dikang:Joined the channel
Wed Sep 14 19:06:56 2016  mkjellman:you mean change the population to fixed(100) ?
Wed Sep 14 19:07:04 2016  tjake:yes
Wed Sep 14 19:07:09 2016  mkjellman:yeah i've got pretty good hardware ;)
Wed Sep 14 19:07:12 2016  tjake:thats means 100 partition keys
Wed Sep 14 19:07:27 2016  tjake:in the system
Wed Sep 14 19:08:17 2016  mkjellman:WARNING: You have defined a schema that permits very large partitions (1000000000 max rows (>100M))
Wed Sep 14 19:08:17 2016  mkjellman:WARNING: You have defined a schema that permits very large batches (10000000 max rows (>100K)). This may OOM this stress client, or the server.
Wed Sep 14 19:08:22 2016  mkjellman:well now we're getting somewhere :)
Wed Sep 14 19:08:41 2016  driftx:hah
Wed Sep 14 19:08:42 2016  tjake:whats the insert look like?
Wed Sep 14 19:08:58 2016  tjake:insert section?
Wed Sep 14 19:08:58 2016  mkjellman:you mean "Generating batches with [10..10] partitions and [10000000..10000000] rows (of [10000000000..10000000000] total rows in the partitions)" ?
Wed Sep 14 19:09:05 2016  tjake:in the yaml
Wed Sep 14 19:09:21 2016  mkjellman:i didn't change it from what i had in the pastebin example
Wed Sep 14 19:09:25 2016  tjake:k
Wed Sep 14 19:09:33 2016  mkjellman:insert:
Wed Sep 14 19:09:33 2016  mkjellman: partitions: fixed(10)
Wed Sep 14 19:09:37 2016  mkjellman:batchtype: UNLOGGED
Wed Sep 14 19:09:41 2016  mkjellman:select: fixed(1)/1000
Wed Sep 14 19:10:00 2016  tjake:make partitions: fixed(1)
Wed Sep 14 19:10:25 2016  mkjellman: PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
Wed Sep 14 19:10:26 2016  mkjellman:56733 mkjellma 20 0 46.9g 27g 14m S 1793.5 22.2 36:23.23 java
Wed Sep 14 19:10:28 2016  tjake:make select: fixed(1)/10000000000
Wed Sep 14 19:10:31 2016  mkjellman:good testing of tim sort going on here.
Wed Sep 14 19:46:07 2016  mkjellman:Joined the channel
Wed Sep 14 19:46:27 2016  mkjellman:sorry tjake -- had some network issues.. bug filed and i'm back :)
Wed Sep 14 19:48:56 2016  mkjellman:sorta sad that we consider >100M "very large partitions"
Wed Sep 14 19:51:32 2016  mkjellman:stress is gc'ing itself to death :(
Wed Sep 14 19:51:39 2016  mkjellman:never finished generating the batches
Wed Sep 14 19:51:43 2016  mkjellman:with a 30GB heap
Wed Sep 14 19:55:35 2016  kohlisankalp:Joined the channel
Wed Sep 14 19:55:51 2016  mkjellman:maybe i should try a 50GB heap? i mean, why not?
Wed Sep 14 20:09:34 2016  tjake:mkjellman: did you see those insert changes I made?
Wed Sep 14 20:09:43 2016  mkjellman:yeah, i did that
Wed Sep 14 20:10:03 2016  tjake:does it output that large batch thing anymore?
Wed Sep 14 20:10:28 2016  mkjellman:http://pastebin.com/qUyZ0L7q
Wed Sep 14 20:10:31 2016  mkjellman:all the threads are doing this
Wed Sep 14 20:11:14 2016  tjake:ugh, yeah its just not able to cope
Wed Sep 14 20:11:22 2016  mkjellman:trying with fixed(1M) for date {cluster} and 25 threads but it's still falling over.
Wed Sep 14 20:12:41 2016  mkjellman:yeah, no good.. i guess let me try 5 threads.. and run multiple instances of stress at the same time instead?
Wed Sep 14 20:14:46 2016  ajasuja:Joined the channel
Wed Sep 14 20:14:48 2016  ob:Joined the channel
Wed Sep 14 20:18:58 2016  mkjellman:this isn't gonna work
Wed Sep 14 20:19:10 2016  mkjellman:i'm gonna just have to write something that generates the stress load iterativly
Wed Sep 14 20:19:14 2016  mkjellman:vs doing everything up front
Wed Sep 14 20:19:35 2016  mkjellman:i can't even do 500,000 and 5 threads with a 50GB heap..
Wed Sep 14 20:26:48 2016  ob:Joined the channel
Wed Sep 14 20:35:17 2016  thobbs:what are we waiting on for the 3.8/3.9 release at this point?
Wed Sep 14 20:35:51 2016  thobbs:I think the outstanding issues were addressed
Wed Sep 14 20:36:17 2016  thobbs:test failures (based on the last report) should now be one flaky test
Wed Sep 14 20:41:19 2016  driftx:maybe that vote you haven't voted on
Wed Sep 14 20:42:25 2016  minimarcel:Joined the channel
Wed Sep 14 20:42:31 2016  thobbs:which one? I don't see any recent votes on dev
Wed Sep 14 20:43:41 2016  driftx:check private
Wed Sep 14 20:44:04 2016  thobbs:ah, didn't realise that was related to releasing
Wed Sep 14 20:44:32 2016  driftx:it will be, if not now, then soon
Wed Sep 14 20:45:25 2016  thobbs:I'd mostly like to see 3.0.9 get released
Wed Sep 14 20:45:40 2016  ptnapoleon:Agreed
Wed Sep 14 20:45:41 2016  thobbs:I assume we're waiting on 3.9 so that we can do them at the same time
Wed Sep 14 20:45:57 2016  ptnapoleon:There aren't really any product-based test failures that i know of, right now
Wed Sep 14 20:46:19 2016  driftx:tjake can do it, if willing
Wed Sep 14 20:59:27 2016  kohlisankalp:Joined the channel
Wed Sep 14 21:01:14 2016  jmckenzie:Joined the channel
Wed Sep 14 21:29:54 2016  clohfink:Joined the channel
Wed Sep 14 21:37:34 2016  adamholmberg:Joined the channel
Wed Sep 14 21:51:32 2016  mstepura:Joined the channel
Wed Sep 14 22:16:00 2016  clohfink:Joined the channel
Wed Sep 14 22:35:12 2016  minimarcel:Joined the channel
Wed Sep 14 23:22:18 2016  kohlisankalp:Joined the channel
Wed Sep 14 23:32:58 2016  mstepura:Joined the channel

Comments