http://redmine.nehome.net/redmine/projects/nosql/wiki/HowTo_-_YCSB_for_Cassandra
HowTo - YCSB for Cassandra
Install/Configuration
설치/설정이라고 하지만, 특별한게 없다. Cassandra(v1.1.4) 및 JDK는 설치 되어 있다는 전제하에서 YCSB를 다운로드 하고 압축을 해제하는 것으로 완료.
<진행 환경>- Machines: node1(172.21.81.139), node2(172.21.81.127)
- OS: Ubuntu 11.10 (oneiric) x86_64
- Cassandra: 1.1.4 (설치 위치: /usr/local/cassandra)
- JDK Version: "1.6.0_27"
- 테스트 작업 위치: /usr/local/src
# cd /usr/local/src/
# wget https://github.com/downloads/brianfrankcooper/YCSB/ycsb-0.1.4.tar.gz
# tar zxvf ycsb-0.1.4.tar.gz
YCSB 테스트를 위한 Cassandra 준비
YCSB & Cassandra 테스트를 위해서, Cassandra에 "usertable"라는 Keyspace에, data라는 Column Family가 필요하다.
KS/CF 생성
# /usr/local/cassandra/bin/cassandra-cli
# create keyspace usertable;
# use usertable;
# create column family data;
YCSB 테스트 실행
ycsb에 포함된 workload 들을 이용해 간단한 테스트를 실습을 수행 해본다.
ycsb의 Usage 및 Commands/Options 확인
# cd /usr/local/src/ycsb-0.1.4
# ./bin/ycsb
Usage: ./bin/ycsb command database [options]
Commands:
load Execute the load phase
run Execute the transaction phase
shell Interactive mode
Databases:
basic https://github.com/brianfrankcooper/YCSB/tree/master/basic
cassandra-10 https://github.com/brianfrankcooper/YCSB/tree/master/cassandra
cassandra-7 https://github.com/brianfrankcooper/YCSB/tree/master/cassandra
cassandra-8 https://github.com/brianfrankcooper/YCSB/tree/master/cassandra
gemfire https://github.com/brianfrankcooper/YCSB/tree/master/gemfire
hbase https://github.com/brianfrankcooper/YCSB/tree/master/hbase
infinispan https://github.com/brianfrankcooper/YCSB/tree/master/infinispan
jdbc https://github.com/brianfrankcooper/YCSB/tree/master/jdbc
mapkeeper https://github.com/brianfrankcooper/YCSB/tree/master/mapkeeper
mongodb https://github.com/brianfrankcooper/YCSB/tree/master/mongodb
nosqldb https://github.com/brianfrankcooper/YCSB/tree/master/nosqldb
redis https://github.com/brianfrankcooper/YCSB/tree/master/redis
voldemort https://github.com/brianfrankcooper/YCSB/tree/master/voldemort
Options:
-P file Specify workload file
-p key=value Override workload property
-s Print status to stderr
-target n Target ops/sec (default: unthrottled)
-threads n Number of client threads (default: 1)
Workload Files:
There are various predefined workloads under workloads/ directory.
See https://github.com/brianfrankcooper/YCSB/wiki/Core-Properties
for the list of workload properties.
Load 테스트 수행 (Execute the load phase: 초기 데이타 등록)
- Cassandra 1.1.4이므로, Database로 cassandra-10을 선택.
- 테스트 노드로 2개 장비 모두 사용. (hosts=172.21.81.139,172.21.81.127)
- 기타 추가 옵션 지정은 hosts와 마찬가지로 "-p" 옵션을 이용해, key=value형태로 지정/추가 해준다.
- Cassandra Database에 관련된 특화 옵션은 Usage에서 확인되는 git 웹페이지에서 확인 가능. (하단 <부록> 섹션 참고)
# ./bin/ycsb load cassandra-10 -P workloads/workloada -p hosts=172.21.81.139,172.21.81.127 -p cassandra.connectionretries=1 -p cassandra.operationretries=1
java -cp /usr/local/src/ycsb-0.1.4/slf4j-jdk14-1.7.2.jar:/usr/local/src/ycsb-0.1.4/slf4j-api-1.7.2.jar:/usr/local/src/ycsb-0.1.4/hbase-binding/conf:/usr/local/src/ycsb-0.1.4/voldemort-binding/conf:/usr/local/src/ycsb-0.1.4/nosqldb-binding/conf:/usr/local/src/ycsb-0.1.4/gemfire-binding/conf:/usr/local/src/ycsb-0.1.4/core/lib/core-0.1.4.jar:/usr/local/src/ycsb-0.1.4/cassandra-binding/lib/cassandra-binding-0.1.4.jar:/usr/local/src/ycsb-0.1.4/jdbc-binding/conf:/usr/local/src/ycsb-0.1.4/infinispan-binding/conf com.yahoo.ycsb.Client -db com.yahoo.ycsb.db.CassandraClient10 -P workloads/cassandra -p hosts=172.21.81.139 -p cassandra.connectionretries=1 -p cassandra.operationretries=1 -load
YCSB Client 0.1
Command line: -db com.yahoo.ycsb.db.CassandraClient10 -P workloads/cassandra -p hosts=172.21.81.139 -p cassandra.connectionretries=1 -p cassandra.operationretries=1 -load
Loading workload...
Starting test.
[OVERALL], RunTime(ms), 8666.0
[OVERALL], Throughput(ops/sec), 1153.9349180706208
[INSERT], Operations, 10000
[INSERT], AverageLatency(us), 820.3828
[INSERT], MinLatency(us), 147
[INSERT], MaxLatency(us), 35981
[INSERT], 95thPercentileLatency(ms), 1
[INSERT], 99thPercentileLatency(ms), 2
[INSERT], Return=0, 10000
[INSERT], 0, 6016
[INSERT], 1, 3877
[INSERT], 2, 47
[INSERT], 3, 30
[INSERT], 4, 5
[INSERT], 5, 8
[INSERT], 6, 4
[INSERT], 7, 1
[INSERT], 8, 2
[INSERT], 9, 4
[INSERT], 10, 2
[INSERT], 11, 0
[INSERT], 12, 0
[INSERT], 13, 0
[INSERT], 14, 0
[INSERT], 15, 1
[INSERT], 16, 0
[INSERT], 17, 0
[INSERT], 18, 0
[INSERT], 19, 0
[INSERT], 20, 1
[INSERT], 21, 0
[INSERT], 22, 0
[INSERT], 23, 0
[INSERT], 24, 0
[INSERT], 25, 1
[INSERT], 26, 0
[INSERT], 27, 0
[INSERT], 28, 0
.
.
(중략)
.
.
[INSERT], 987, 0
[INSERT], 988, 0
[INSERT], 989, 0
[INSERT], 990, 0
[INSERT], 991, 0
[INSERT], 992, 0
[INSERT], 993, 0
[INSERT], 994, 0
[INSERT], 995, 0
[INSERT], 996, 0
[INSERT], 997, 0
[INSERT], 998, 0
[INSERT], 999, 0
[INSERT], >1000, 0
Run 테스트 수행 (Execute the transaction phase)
# ./bin/ycsb run cassandra-10 -P workloads/cassandra -p hosts=172.21.81.139,172.21.81.127 -p cassandra.connectionretries=1 -p cassandra.operationretries=1
java -cp /usr/local/src/ycsb-0.1.4/slf4j-jdk14-1.7.2.jar:/usr/local/src/ycsb-0.1.4/slf4j-api-1.7.2.jar:/usr/local/src/ycsb-0.1.4/hbase-binding/conf:/usr/local/src/ycsb-0.1.4/voldemort-binding/conf:/usr/local/src/ycsb-0.1.4/nosqldb-binding/conf:/usr/local/src/ycsb-0.1.4/gemfire-binding/conf:/usr/local/src/ycsb-0.1.4/core/lib/core-0.1.4.jar:/usr/local/src/ycsb-0.1.4/cassandra-binding/lib/cassandra-binding-0.1.4.jar:/usr/local/src/ycsb-0.1.4/jdbc-binding/conf:/usr/local/src/ycsb-0.1.4/infinispan-binding/conf com.yahoo.ycsb.Client -db com.yahoo.ycsb.db.CassandraClient10 -P workloads/cassandra -p hosts=172.21.81.139,172.21.81.127 -p cassandra.connectionretries=1 -p cassandra.operationretries=1 -t
YCSB Client 0.1
Command line: -db com.yahoo.ycsb.db.CassandraClient10 -P workloads/cassandra -p hosts=172.21.81.139,172.21.81.127 -p cassandra.connectionretries=1 -p cassandra.operationretries=1 -t
Loading workload...
Starting test.
[OVERALL], RunTime(ms), 13476.0
[OVERALL], Throughput(ops/sec), 742.0599584446423
[INSERT], Operations, 5039
[INSERT], AverageLatency(us), 1280.7376463584044
[INSERT], MinLatency(us), 259
[INSERT], MaxLatency(us), 202840
[INSERT], 95thPercentileLatency(ms), 2
[INSERT], 99thPercentileLatency(ms), 4
[INSERT], Return=0, 5039
[INSERT], 0, 2774
[INSERT], 1, 1840
[INSERT], 2, 289
[INSERT], 3, 66
[INSERT], 4, 20
[INSERT], 5, 10
[INSERT], 6, 5
[INSERT], 7, 6
[INSERT], 8, 5
[INSERT], 9, 3
[INSERT], 10, 6
[INSERT], 11, 1
[INSERT], 12, 2
[INSERT], 13, 2
[INSERT], 14, 2
[INSERT], 15, 1
[INSERT], 16, 0
[INSERT], 17, 1
[INSERT], 18, 1
[INSERT], 19, 0
[INSERT], 20, 1
[INSERT], 21, 0
[INSERT], 22, 0
[INSERT], 23, 1
[INSERT], 24, 0
[INSERT], 25, 1
[INSERT], 26, 0
.
.
(중략)
.
.
[INSERT], 994, 0
[INSERT], 995, 0
[INSERT], 996, 0
[INSERT], 997, 0
[INSERT], 998, 0
[INSERT], 999, 0
[INSERT], >1000, 0
[READ], Operations, 4961
[READ], AverageLatency(us), 1323.8699858899415
[READ], MinLatency(us), 298
[READ], MaxLatency(us), 61646
[READ], 95thPercentileLatency(ms), 2
[READ], 99thPercentileLatency(ms), 4
[READ], Return=0, 4961
[READ], 0, 2226
[READ], 1, 2260
[READ], 2, 362
[READ], 3, 57
[READ], 4, 13
[READ], 5, 7
[READ], 6, 7
[READ], 7, 7
[READ], 8, 4
[READ], 9, 1
[READ], 10, 3
[READ], 11, 2
[READ], 12, 3
[READ], 13, 3
[READ], 14, 1
[READ], 15, 0
[READ], 16, 2
[READ], 17, 0
.
.
(중략)
.
.
[READ], 989, 0
[READ], 990, 0
[READ], 991, 0
[READ], 992, 0
[READ], 993, 0
[READ], 994, 0
[READ], 995, 0
[READ], 996, 0
[READ], 997, 0
[READ], 998, 0
[READ], 999, 0
[READ], >1000, 0
workload
기본적으로 제공되는 workload는 하기와 같이 6가지가 제공된다.
(각각이 추구하는 목적은 링크 https://github.com/brianfrankcooper/YCSB/wiki/Core-Workloads 참조)
- workloada
- workloadb
- workloadc
- workloadd
- workloade
- workloadf
workload 예제 설명
예제로 workloadf를 살펴보자
recordcount=1000
operationcount=1000
workload=com.yahoo.ycsb.workloads.CoreWorkload
readallfields=true
readproportion=0.5
updateproportion=0
scanproportion=0
insertproportion=0
readmodifywriteproportion=0.5
requestdistribution=zipfian
위와 같은 내용을 담고 있으며, 각각의 역할을 명칭에서 드러난다.
[Note] File 참조가 싫으면, 하기와 유사하게 "-p" 옵션을 이용해 하나 하나 지정해줘도 된다.
예를 들어 하기와 같이, 각자의 테스트 목적에 부합되게 설정하여 사용하면 될 것이다.
# ./bin/ycsb \
run \
cassandra-10 \
-s \
-p recordcount=10000 \
-p operationcount=10000 \
-p workload=com.yahoo.ycsb.workloads.CoreWorkload \
-p readallfields=true \
-p readproportion=0.5 \
-p updateproportion=0 \
-p scanproportion=0 \
-p insertproportion=0.5 \
-p readmodifywriteproportion=0 \
-p requestdistribution=zipfian \
-p hosts=172.21.81.139,172.21.81.127 \
-p cassandra.connectionretries=1 \
-p cassandra.operationretries=1 \
-p cassandra.readconsistencylevel=ALL \
-p cassandra.writeconsistencylevel=ALL \
-p cassandra.deleteconsistencylevel=ALL \
-threads 10
- 전체 몇개의 Count로 테스트를 할 것인지...
- 전체 테스트 Count에서 read/update/scan/insert/readmodifywrite의 비율은 어떻게 할 것인지...
부록
Cassandra Database 특화 옵션
Source:https://github.com/brianfrankcooper/YCSB/blob/master/cassandra/src/main/java/com/yahoo/ycsb/db/CassandraClient10.java
public static final String CONNECTION_RETRY_PROPERTY = "cassandra.connectionretries";
public static final String CONNECTION_RETRY_PROPERTY_DEFAULT = "300";
public static final String OPERATION_RETRY_PROPERTY = "cassandra.operationretries";
public static final String OPERATION_RETRY_PROPERTY_DEFAULT = "300";
public static final String USERNAME_PROPERTY = "cassandra.username";
public static final String PASSWORD_PROPERTY = "cassandra.password";
public static final String COLUMN_FAMILY_PROPERTY = "cassandra.columnfamily";
public static final String COLUMN_FAMILY_PROPERTY_DEFAULT = "data";
public static final String READ_CONSISTENCY_LEVEL_PROPERTY = "cassandra.readconsistencylevel";
public static final String READ_CONSISTENCY_LEVEL_PROPERTY_DEFAULT = "ONE";
public static final String WRITE_CONSISTENCY_LEVEL_PROPERTY = "cassandra.writeconsistencylevel";
public static final String WRITE_CONSISTENCY_LEVEL_PROPERTY_DEFAULT = "ONE";
public static final String SCAN_CONSISTENCY_LEVEL_PROPERTY = "cassandra.scanconsistencylevel";
public static final String SCAN_CONSISTENCY_LEVEL_PROPERTY_DEFAULT = "ONE";
public static final String DELETE_CONSISTENCY_LEVEL_PROPERTY = "cassandra.deleteconsistencylevel";
public static final String DELETE_CONSISTENCY_LEVEL_PROPERTY_DEFAULT = "ONE";
http://zopub.com/download?doc_id=519
EC2-Latency.xlsx
