SCYLLA: NoSQL at Ludicrous Speed主讲人：ScyllaDB软件工程师 贺俊Today we will cover:+ Intro: Who we are, what we do, who uses it+ Why we started ScyllaDB+ Why should you care+ How we made design decisions to achieve no-compromise performance and availability+Founded by KVM hypervisor creators+Q2 2014 - Pivot to the database world+Q3 2015 - Decloak during Cassandra Summit 2015, Beta+Q1 2016 - General Availability+Q3 2016 - First Scylla Summit: 100+ Attendees+Q1 2017 - Completed B round+$25MM in funding+HQs: Palo Alto, CA; Herzelia, Israel+42+ employees, hiring!IntroductionWhy ?@#$%$$%^?Scylla benchmark by Samsungop/sWhat we do: Scylla, towards the best NoSQL+> 1 million OPS per node+< 1ms 99% latency+Auto tuned+Scale up and out+Open source+Large community (piggyback on Cassandra)+Blends in the ecosystem- Spark, Presto, time series, search, ..Where Scylla is deployed?Today we will cover:+ Intro: Who we are, what we do, who uses it+ Why we started ScyllaDB+ Why should you care+ How we made design decisions to achieve no-compromise performance and availabilityWhy we started Scylla?+Originally it was about performance/efficiency only+Over time, we understood we can deliver more:+SLA between background and foreground tasks+Work well on any given hardware {back pressure}+Deliver consistent, low 99th percentile latency+Reduction in admin effort+Low latency under the face of failures (hot cache load balancing)+High observabilityCassandra ScyllaThroughput: Cannot utilize multi-core efficientlyScales linearly - shard-per-coreLatency:High due to Java and JVM’s GCLow and consistent - own cacheComplexity:Intricate tuning and configurationAuto tuned, dynamic schedulingAdmin:Maintenance impacts performanceSLA guarantee for admin vs servingToday we will cover:+ Intro: Who we are, what we do, who uses it+ Why we started ScyllaDB+ Why should you care+ How we made design decisions to achieve no-compromise performance and availabilityCase study: Document column family•Outbrain is the world’s largest content discovery platform.•Over 557 million unique visitors from across the globe.•250 billion personalized content recommendations every month.Outbrain: Cassandra plus Memcache•First read from memcached, go to Cassandra on misses.•Pain: 1) Stale data from cache 2) Complexity 3) Cold cache -> C* gets full volumeReadMicroserviceWrite ProcessScylla/Cassandra side by side deployment•Writes are written in parallel to C* and Scylla•Reads are done in parallel:1) Memcached + Cassandra 2) Scylla (no cache at all)ReadMicroserviceWrite ProcessScylla (w/o cache) vs Cassandra + Memcached15ScyllaCassandraDiff%Requests/Minute12,000,000500,000(memcache handles 11,500,000)24XAVG Latency4 ms8 ms2XMax Latency8 ms35 ms3.5XHardware9 machines30+9 machines4.3XCassandra’s latencyScylla’s latencyWhat does it mean for a non Cassandra user?+ Throughput, latency and scale benefits+ Wide range of big data integration: {Kariosdb, Spark, JanusGraph, Presto, Kafka, Elastic}+ Best HA/DR in the industry. + Stop using caches in front of the database+ Consolidate HBase, Redis, MySQL, Mongo and othersAssorted QuotesToday we will cover:+ Intro: Who we are, what we do, who uses it+ Why we started ScyllaDB+ Why should you care+ How we made design decisions to achieve no-compromise performance and availabilityDesign decisions: #1 The trivials•SSTable file format•Configuration file format•CQL language•CQL native protocol•JMX management protocol•Management command lineDesign decisions: #2 Compatibility Double cluster - Migration w/o downtimeAppCassandraScyllaCQLproxyDesign decisions: #3 All things asyncDesign decisions: #4 Shard per coreThreads ShardsSCYLLA DB: Network ComparisonKernelCassandraTCP/IPSchedulerqueuequeuequeuequeuequeuethreadsNICQueuesKernelTraditional stackScylla sharded stackMemoryApplicationTCP/IPTask Schedulerqueuequeuequeuequeuequeuesmp queueNICQueueDPDKKernel (isn’t involved)UserspaceApplicationTCP/IPTask Schedulerqueuequeuequeuequeuequeuesmp queueNICQueueDPDKKernel (isn’t involved)UserspaceApplicationTCP/IPTask Schedulerqueuequeuequeuequeuequeuesmp queueNICQueueDPDKKernel (isn’t involved)UserspaceCoreDatabaseTask Schedulerqueuequeuequeuequeuequeuesmp queueNICQueueUserspaceScylla has its own task schedulerTraditional stackScylla’s stackPromiseTaskPromiseTaskPromiseTaskPromiseTaskCPUPromiseTaskPromiseTaskPromiseTaskPromiseTaskCPUPromiseTaskPromiseTaskPromiseTaskPromiseTaskCPUPromiseTaskPromiseTaskPromiseTaskPromiseTaskCPUPromiseTaskPromiseTaskPromiseTaskPromiseTaskCPUPromise is a pointer to eventually computed valueTask is a pointer to a lambda functionSchedulerCPUSchedulerCPUSchedulerCPUSchedulerCPUSchedulerCPUThreadStackThreadStackThreadStackThreadStackThreadStackThreadStackThreadStackThreadStackThread is a function pointerStack is a byte array from 64k to megabytesSCYLLA IS DIFFERENTp DMAp Log structuredmerge treep DBaware cachep Userspace I/Oschedulerp NUMA friendlyp Log structured allocatorp Zero copyp Thread per corep Lock-freep Task schedulerp Reactor program...
发表评论取消回复