Displaying #cassandra-dev/2017-06-16.log:

Fri Jun 16 00:21:43 2017  RussSpitzer:Joined the channel
Fri Jun 16 01:01:57 2017  JayZhuang:Joined the channel
Fri Jun 16 01:02:51 2017  RussSpitzer:Joined the channel
Fri Jun 16 01:03:52 2017  adamholmberg:Joined the channel
Fri Jun 16 02:18:45 2017  mstepura:Joined the channel
Fri Jun 16 02:33:34 2017  mebigfatguy:Joined the channel
Fri Jun 16 03:58:48 2017  mebigfatguy:Joined the channel
Fri Jun 16 04:04:18 2017  adamholmberg:Joined the channel
Fri Jun 16 04:04:36 2017  iskm:Joined the channel
Fri Jun 16 04:24:42 2017  kohlisankalp:Joined the channel
Fri Jun 16 04:37:29 2017  kohlisankalp:Joined the channel
Fri Jun 16 04:48:28 2017  mebigfatguy:Joined the channel
Fri Jun 16 05:35:32 2017  adamholmberg:Joined the channel
Fri Jun 16 05:42:23 2017  mstepura:Joined the channel
Fri Jun 16 06:52:00 2017  mstepura:Joined the channel
Fri Jun 16 07:17:17 2017  sedulam:Joined the channel
Fri Jun 16 07:49:39 2017  kvaster:Joined the channel
Fri Jun 16 08:06:32 2017  adamholmberg:Joined the channel
Fri Jun 16 08:20:48 2017  iskm:Joined the channel
Fri Jun 16 08:25:37 2017  sedulam:Joined the channel
Fri Jun 16 09:00:37 2017  dipankar12:Joined the channel
Fri Jun 16 09:02:15 2017  dipankar12:join
Fri Jun 16 09:02:23 2017  dipankar12:hello
Fri Jun 16 09:59:45 2017  mebigfatguy:Joined the channel
Fri Jun 16 10:07:48 2017  adamholmberg:Joined the channel
Fri Jun 16 10:29:50 2017  sedulam:Joined the channel
Fri Jun 16 11:09:07 2017  adamholmberg:Joined the channel
Fri Jun 16 12:08:23 2017  iksaif:hey, I'm looking for a way to apply a function to all my cells when they move from the memtable to the disk (it's a timeseries, I'd like to do double-delta encoding there)
Fri Jun 16 12:08:30 2017  iksaif:do we have any hooks to do that already ?
Fri Jun 16 12:09:31 2017  iksaif:ideally something like UDF/UDA, but on the write path
Fri Jun 16 12:26:29 2017  gaYak:You can't
Fri Jun 16 12:36:32 2017  adamholmberg:Joined the channel
Fri Jun 16 12:53:32 2017  iksaif:so the only option (currently) is a custom compactio strategy ?
Fri Jun 16 12:58:31 2017  gaYak:No, even that would fail if you later update a single value / receive something from other node
Fri Jun 16 12:58:40 2017  gaYak:As your DoD depends on each other
Fri Jun 16 12:59:06 2017  gaYak:The way I solved it in the Hawkular-Metrics is to write first to temporary tables and then compact those temporary tables to another table
Fri Jun 16 12:59:26 2017  makrusak:Joined the channel
Fri Jun 16 12:59:30 2017  gaYak:And of course create a serialization on top of the rows (so one cell packs a large binary containing large amount of cells)
Fri Jun 16 12:59:46 2017  iksaif:True, read-repairs would get in the way
Fri Jun 16 12:59:55 2017  iksaif:and other stuff..
Fri Jun 16 13:00:02 2017  gaYak:Yeah, it's going to be hack no matter what you do ;)
Fri Jun 16 13:01:31 2017  iksaif:To do it efficiently I could take a snapshot periodically, and read directly from the snapshot to re-"compress" the data
Fri Jun 16 13:02:31 2017  gaYak:Yeah, if you can access the snapshots from your application code (containers are a pain in this sense)
Fri Jun 16 13:02:45 2017  gaYak:Other idea is to write these to memory first and only keep a WAL in the Cassandra
Fri Jun 16 13:02:57 2017  gaYak:(that's next on my list of things to do ..)
Fri Jun 16 13:04:16 2017  iksaif:currently it goes directly to Cassandra to avoid having to read from different places (and because we can't afford more than two minutes of delays)
Fri Jun 16 13:04:37 2017  iksaif:which sucks when the periodicity of your points is 1 minute, there is close to no batching that can be done
Fri Jun 16 13:05:02 2017  gaYak:Yeah, we use the secondary gateway to combine points from different places
Fri Jun 16 13:05:04 2017  gaYak:To a single stream
Fri Jun 16 13:05:16 2017  gaYak:But of course that requires serialization / deserialization and abstracts the Cassandra away
Fri Jun 16 13:06:00 2017  iksaif:In this case it's for Graphite/BigGaphite .. in theory Carbon provides read API, and Graphite can act as a gateway
Fri Jun 16 13:06:28 2017  iksaif:but .. carbon isn't really stable enough for us to make it do more things
Fri Jun 16 13:08:03 2017  gaYak:The BigGraphite itself can't do that?
Fri Jun 16 13:08:28 2017  iksaif:BigGraphite is just a plugin for carbon to write in Cassandra and for Graphite to read from Cassandra
Fri Jun 16 13:09:12 2017  gaYak:Well, then the temp table stuff is one option still.. you just issue two reads to Cassandra and merge the results. But of course that requires state management and such stuff
Fri Jun 16 13:09:47 2017  iksaif:yep, my main issue isn't even size, it's that having a lot of Cells adds a lot of overhead
Fri Jun 16 13:09:56 2017  gaYak:Yep
Fri Jun 16 13:10:16 2017  gaYak:We found a huge improvement in performance when reading the compressed blocks we write (just because its less cells to read)
Fri Jun 16 13:10:47 2017  iksaif:also for repairs / compactions ..
Fri Jun 16 13:15:39 2017  gaYak:Hmph, Java-driver didn't like my idea of making a query with time >= minTimeuuid(Date) ..
Fri Jun 16 13:38:59 2017  aboudreault:Joined the channel
Fri Jun 16 13:58:15 2017  iskm:Joined the channel
Fri Jun 16 14:54:32 2017  mstepura:Joined the channel
Fri Jun 16 14:57:00 2017  aboudreault:Joined the channel
Fri Jun 16 15:06:00 2017  adamholm_:Joined the channel
Fri Jun 16 15:20:44 2017  mstepura:Joined the channel
Fri Jun 16 15:28:29 2017  aboudreault:Joined the channel
Fri Jun 16 15:28:34 2017  adamholmberg:Joined the channel
Fri Jun 16 15:32:45 2017  adamholm_:Joined the channel
Fri Jun 16 15:35:30 2017  kohlisankalp:Joined the channel
Fri Jun 16 15:55:39 2017  kohlisankalp:Joined the channel
Fri Jun 16 16:26:16 2017  kohlisankalp:Joined the channel
Fri Jun 16 16:26:40 2017  adamholmberg:Joined the channel
Fri Jun 16 16:35:08 2017  adamholmberg:Joined the channel
Fri Jun 16 16:41:58 2017  JayZhuang:Joined the channel
Fri Jun 16 16:46:15 2017  adamholmberg:Joined the channel
Fri Jun 16 16:49:24 2017  adamholm_:Joined the channel
Fri Jun 16 16:57:29 2017  sedulam_:Joined the channel
Fri Jun 16 17:02:48 2017  sedulam_:Hi all! I found that the option "duration" is missing from the help list in the cassandra-stress tool. The option is recognized, but it doesn't show after using the help option, nor is it in the in-tree documentation. Is this reason enough for opening a JIRA? I'll fix it, but I'm unsure if the scope is enough for a ticket by itself.
Fri Jun 16 17:02:49 2017  kohlisankalp:Joined the channel
Fri Jun 16 17:26:14 2017  EdwardCapriolo:Joined the channel
Fri Jun 16 17:30:01 2017  mstepura:Joined the channel
Fri Jun 16 17:35:03 2017  sedulam_:Joined the channel
Fri Jun 16 18:02:55 2017  kohlisan_:Joined the channel
Fri Jun 16 18:26:59 2017  sedulam_:nvm, found it :)
Fri Jun 16 19:12:01 2017  sedulam_:Joined the channel
Fri Jun 16 19:12:06 2017  adamholmberg:Joined the channel
Fri Jun 16 19:13:32 2017  kohlisankalp:Joined the channel
Fri Jun 16 20:24:05 2017  mebigfatguy:Joined the channel
Fri Jun 16 20:30:17 2017  adamholmberg:Joined the channel
Fri Jun 16 20:37:32 2017  adamholmberg:Joined the channel
Fri Jun 16 21:02:33 2017  JayZhuang:Joined the channel
Fri Jun 16 21:03:15 2017  JayZhuang:Joined the channel
Fri Jun 16 21:20:11 2017  kohlisankalp:Joined the channel
Fri Jun 16 21:27:49 2017  mstepura:Joined the channel
Fri Jun 16 21:46:17 2017  JayZhuang1:Joined the channel
Fri Jun 16 22:32:04 2017  kohlisankalp:Joined the channel
Fri Jun 16 22:52:10 2017  mstepura:Joined the channel
Fri Jun 16 23:27:42 2017  ASFBot:Joined the channel
Fri Jun 16 23:43:03 2017  kohlisankalp:Joined the channel
Fri Jun 16 23:43:23 2017  kohlisankalp:Joined the channel
Fri Jun 16 23:55:17 2017  cassci:Joined the channel

Comments