HowTo - YCSB (for Cassandra)

카테고리 없음 2013. 1. 7. 00:34

http://redmine.nehome.net/redmine/projects/nosql/wiki/HowTo_-_YCSB_for_Cassandra

HowTo - YCSB for Cassandra

Install/Configuration

설치/설정이라고 하지만, 특별한게 없다. Cassandra(v1.1.4) 및 JDK는 설치 되어 있다는 전제하에서 YCSB를 다운로드 하고 압축을 해제하는 것으로 완료.

<진행 환경>

Machines: node1(172.21.81.139), node2(172.21.81.127)
OS: Ubuntu 11.10 (oneiric) x86_64
Cassandra: 1.1.4 (설치 위치: /usr/local/cassandra)
JDK Version: "1.6.0_27"
테스트 작업 위치: /usr/local/src

# cd /usr/local/src/
# wget https://github.com/downloads/brianfrankcooper/YCSB/ycsb-0.1.4.tar.gz
# tar zxvf ycsb-0.1.4.tar.gz

YCSB 테스트를 위한 Cassandra 준비

YCSB & Cassandra 테스트를 위해서, Cassandra에 "usertable"라는 Keyspace에, data라는 Column Family가 필요하다.

KS/CF 생성

# /usr/local/cassandra/bin/cassandra-cli
# create keyspace usertable;
# use usertable;
# create column family data;

YCSB 테스트 실행

ycsb에 포함된 workload 들을 이용해 간단한 테스트를 실습을 수행 해본다.

ycsb의 Usage 및 Commands/Options 확인

# cd /usr/local/src/ycsb-0.1.4
# ./bin/ycsb
Usage: ./bin/ycsb command database [options]

Commands:
    load          Execute the load phase
    run           Execute the transaction phase
    shell         Interactive mode

Databases:
    basic         https://github.com/brianfrankcooper/YCSB/tree/master/basic
    cassandra-10  https://github.com/brianfrankcooper/YCSB/tree/master/cassandra
    cassandra-7   https://github.com/brianfrankcooper/YCSB/tree/master/cassandra
    cassandra-8   https://github.com/brianfrankcooper/YCSB/tree/master/cassandra
    gemfire       https://github.com/brianfrankcooper/YCSB/tree/master/gemfire
    hbase         https://github.com/brianfrankcooper/YCSB/tree/master/hbase
    infinispan    https://github.com/brianfrankcooper/YCSB/tree/master/infinispan
    jdbc          https://github.com/brianfrankcooper/YCSB/tree/master/jdbc
    mapkeeper     https://github.com/brianfrankcooper/YCSB/tree/master/mapkeeper
    mongodb       https://github.com/brianfrankcooper/YCSB/tree/master/mongodb
    nosqldb       https://github.com/brianfrankcooper/YCSB/tree/master/nosqldb
    redis         https://github.com/brianfrankcooper/YCSB/tree/master/redis
    voldemort     https://github.com/brianfrankcooper/YCSB/tree/master/voldemort

Options:
    -P file       Specify workload file
    -p key=value  Override workload property
    -s            Print status to stderr
    -target n     Target ops/sec (default: unthrottled)
    -threads n    Number of client threads (default: 1)

Workload Files:
    There are various predefined workloads under workloads/ directory.
    See https://github.com/brianfrankcooper/YCSB/wiki/Core-Properties
    for the list of workload properties.

Load 테스트 수행 (Execute the load phase: 초기 데이타 등록)

Cassandra 1.1.4이므로, Database로 cassandra-10을 선택.
테스트 노드로 2개 장비 모두 사용. (hosts=172.21.81.139,172.21.81.127)
기타 추가 옵션 지정은 hosts와 마찬가지로 "-p" 옵션을 이용해, key=value형태로 지정/추가 해준다.
Cassandra Database에 관련된 특화 옵션은 Usage에서 확인되는 git 웹페이지에서 확인 가능. (하단 <부록> 섹션 참고)

# ./bin/ycsb load cassandra-10 -P workloads/workloada -p hosts=172.21.81.139,172.21.81.127 -p cassandra.connectionretries=1 -p cassandra.operationretries=1
java -cp /usr/local/src/ycsb-0.1.4/slf4j-jdk14-1.7.2.jar:/usr/local/src/ycsb-0.1.4/slf4j-api-1.7.2.jar:/usr/local/src/ycsb-0.1.4/hbase-binding/conf:/usr/local/src/ycsb-0.1.4/voldemort-binding/conf:/usr/local/src/ycsb-0.1.4/nosqldb-binding/conf:/usr/local/src/ycsb-0.1.4/gemfire-binding/conf:/usr/local/src/ycsb-0.1.4/core/lib/core-0.1.4.jar:/usr/local/src/ycsb-0.1.4/cassandra-binding/lib/cassandra-binding-0.1.4.jar:/usr/local/src/ycsb-0.1.4/jdbc-binding/conf:/usr/local/src/ycsb-0.1.4/infinispan-binding/conf com.yahoo.ycsb.Client -db com.yahoo.ycsb.db.CassandraClient10 -P workloads/cassandra -p hosts=172.21.81.139 -p cassandra.connectionretries=1 -p cassandra.operationretries=1 -load
YCSB Client 0.1
Command line: -db com.yahoo.ycsb.db.CassandraClient10 -P workloads/cassandra -p hosts=172.21.81.139 -p cassandra.connectionretries=1 -p cassandra.operationretries=1 -load
Loading workload...
Starting test.
[OVERALL], RunTime(ms), 8666.0
[OVERALL], Throughput(ops/sec), 1153.9349180706208
[INSERT], Operations, 10000
[INSERT], AverageLatency(us), 820.3828
[INSERT], MinLatency(us), 147
[INSERT], MaxLatency(us), 35981
[INSERT], 95thPercentileLatency(ms), 1
[INSERT], 99thPercentileLatency(ms), 2
[INSERT], Return=0, 10000
[INSERT], 0, 6016
[INSERT], 1, 3877
[INSERT], 2, 47
[INSERT], 3, 30
[INSERT], 4, 5
[INSERT], 5, 8
[INSERT], 6, 4
[INSERT], 7, 1
[INSERT], 8, 2
[INSERT], 9, 4
[INSERT], 10, 2
[INSERT], 11, 0
[INSERT], 12, 0
[INSERT], 13, 0
[INSERT], 14, 0
[INSERT], 15, 1
[INSERT], 16, 0
[INSERT], 17, 0
[INSERT], 18, 0
[INSERT], 19, 0
[INSERT], 20, 1
[INSERT], 21, 0
[INSERT], 22, 0
[INSERT], 23, 0
[INSERT], 24, 0
[INSERT], 25, 1
[INSERT], 26, 0
[INSERT], 27, 0
[INSERT], 28, 0
.
.
(중략)
.
.
[INSERT], 987, 0
[INSERT], 988, 0
[INSERT], 989, 0
[INSERT], 990, 0
[INSERT], 991, 0
[INSERT], 992, 0
[INSERT], 993, 0
[INSERT], 994, 0
[INSERT], 995, 0
[INSERT], 996, 0
[INSERT], 997, 0
[INSERT], 998, 0
[INSERT], 999, 0
[INSERT], >1000, 0

Run 테스트 수행 (Execute the transaction phase)

# ./bin/ycsb run cassandra-10 -P workloads/cassandra -p hosts=172.21.81.139,172.21.81.127 -p cassandra.connectionretries=1 -p cassandra.operationretries=1
java -cp /usr/local/src/ycsb-0.1.4/slf4j-jdk14-1.7.2.jar:/usr/local/src/ycsb-0.1.4/slf4j-api-1.7.2.jar:/usr/local/src/ycsb-0.1.4/hbase-binding/conf:/usr/local/src/ycsb-0.1.4/voldemort-binding/conf:/usr/local/src/ycsb-0.1.4/nosqldb-binding/conf:/usr/local/src/ycsb-0.1.4/gemfire-binding/conf:/usr/local/src/ycsb-0.1.4/core/lib/core-0.1.4.jar:/usr/local/src/ycsb-0.1.4/cassandra-binding/lib/cassandra-binding-0.1.4.jar:/usr/local/src/ycsb-0.1.4/jdbc-binding/conf:/usr/local/src/ycsb-0.1.4/infinispan-binding/conf com.yahoo.ycsb.Client -db com.yahoo.ycsb.db.CassandraClient10 -P workloads/cassandra -p hosts=172.21.81.139,172.21.81.127 -p cassandra.connectionretries=1 -p cassandra.operationretries=1 -t
YCSB Client 0.1
Command line: -db com.yahoo.ycsb.db.CassandraClient10 -P workloads/cassandra -p hosts=172.21.81.139,172.21.81.127 -p cassandra.connectionretries=1 -p cassandra.operationretries=1 -t
Loading workload...
Starting test.
[OVERALL], RunTime(ms), 13476.0
[OVERALL], Throughput(ops/sec), 742.0599584446423
[INSERT], Operations, 5039
[INSERT], AverageLatency(us), 1280.7376463584044
[INSERT], MinLatency(us), 259
[INSERT], MaxLatency(us), 202840
[INSERT], 95thPercentileLatency(ms), 2
[INSERT], 99thPercentileLatency(ms), 4
[INSERT], Return=0, 5039
[INSERT], 0, 2774
[INSERT], 1, 1840
[INSERT], 2, 289
[INSERT], 3, 66
[INSERT], 4, 20
[INSERT], 5, 10
[INSERT], 6, 5
[INSERT], 7, 6
[INSERT], 8, 5
[INSERT], 9, 3
[INSERT], 10, 6
[INSERT], 11, 1
[INSERT], 12, 2
[INSERT], 13, 2
[INSERT], 14, 2
[INSERT], 15, 1
[INSERT], 16, 0
[INSERT], 17, 1
[INSERT], 18, 1
[INSERT], 19, 0
[INSERT], 20, 1
[INSERT], 21, 0
[INSERT], 22, 0
[INSERT], 23, 1
[INSERT], 24, 0
[INSERT], 25, 1
[INSERT], 26, 0
.
.
(중략)
.
.
[INSERT], 994, 0
[INSERT], 995, 0
[INSERT], 996, 0
[INSERT], 997, 0
[INSERT], 998, 0
[INSERT], 999, 0
[INSERT], >1000, 0
[READ], Operations, 4961
[READ], AverageLatency(us), 1323.8699858899415
[READ], MinLatency(us), 298
[READ], MaxLatency(us), 61646
[READ], 95thPercentileLatency(ms), 2
[READ], 99thPercentileLatency(ms), 4
[READ], Return=0, 4961
[READ], 0, 2226
[READ], 1, 2260
[READ], 2, 362
[READ], 3, 57
[READ], 4, 13
[READ], 5, 7
[READ], 6, 7
[READ], 7, 7
[READ], 8, 4
[READ], 9, 1
[READ], 10, 3
[READ], 11, 2
[READ], 12, 3
[READ], 13, 3
[READ], 14, 1
[READ], 15, 0
[READ], 16, 2
[READ], 17, 0
.
.
(중략)
.
.
[READ], 989, 0
[READ], 990, 0
[READ], 991, 0
[READ], 992, 0
[READ], 993, 0
[READ], 994, 0
[READ], 995, 0
[READ], 996, 0
[READ], 997, 0
[READ], 998, 0
[READ], 999, 0
[READ], >1000, 0

workload

기본적으로 제공되는 workload는 하기와 같이 6가지가 제공된다.
(각각이 추구하는 목적은 링크 https://github.com/brianfrankcooper/YCSB/wiki/Core-Workloads 참조)

workloada
workloadb
workloadc
workloadd
workloade
workloadf

workload 예제 설명

예제로 workloadf를 살펴보자

recordcount=1000
operationcount=1000
workload=com.yahoo.ycsb.workloads.CoreWorkload

readallfields=true

readproportion=0.5
updateproportion=0
scanproportion=0
insertproportion=0
readmodifywriteproportion=0.5

requestdistribution=zipfian

위와 같은 내용을 담고 있으며, 각각의 역할을 명칭에서 드러난다.
[Note] File 참조가 싫으면, 하기와 유사하게 "-p" 옵션을 이용해 하나 하나 지정해줘도 된다.
예를 들어 하기와 같이, 각자의 테스트 목적에 부합되게 설정하여 사용하면 될 것이다.

# ./bin/ycsb \ 
run \ 
cassandra-10 \
-s \ 
-p recordcount=10000 \
-p operationcount=10000 \
-p workload=com.yahoo.ycsb.workloads.CoreWorkload \
-p readallfields=true \
-p readproportion=0.5 \
-p updateproportion=0 \
-p scanproportion=0 \
-p insertproportion=0.5 \
-p readmodifywriteproportion=0 \
-p requestdistribution=zipfian \
-p hosts=172.21.81.139,172.21.81.127 \
-p cassandra.connectionretries=1 \
-p cassandra.operationretries=1 \
-p cassandra.readconsistencylevel=ALL \
-p cassandra.writeconsistencylevel=ALL \
-p cassandra.deleteconsistencylevel=ALL \
-threads 10

전체 몇개의 Count로 테스트를 할 것인지...
전체 테스트 Count에서 read/update/scan/insert/readmodifywrite의 비율은 어떻게 할 것인지...

부록

Cassandra Database 특화 옵션

Source:https://github.com/brianfrankcooper/YCSB/blob/master/cassandra/src/main/java/com/yahoo/ycsb/db/CassandraClient10.java

  public static final String CONNECTION_RETRY_PROPERTY = "cassandra.connectionretries";
  public static final String CONNECTION_RETRY_PROPERTY_DEFAULT = "300";

  public static final String OPERATION_RETRY_PROPERTY = "cassandra.operationretries";
  public static final String OPERATION_RETRY_PROPERTY_DEFAULT = "300";

  public static final String USERNAME_PROPERTY = "cassandra.username";
  public static final String PASSWORD_PROPERTY = "cassandra.password";

  public static final String COLUMN_FAMILY_PROPERTY = "cassandra.columnfamily";
  public static final String COLUMN_FAMILY_PROPERTY_DEFAULT = "data";

  public static final String READ_CONSISTENCY_LEVEL_PROPERTY = "cassandra.readconsistencylevel";
  public static final String READ_CONSISTENCY_LEVEL_PROPERTY_DEFAULT = "ONE";

  public static final String WRITE_CONSISTENCY_LEVEL_PROPERTY = "cassandra.writeconsistencylevel";
  public static final String WRITE_CONSISTENCY_LEVEL_PROPERTY_DEFAULT = "ONE";

  public static final String SCAN_CONSISTENCY_LEVEL_PROPERTY = "cassandra.scanconsistencylevel";
  public static final String SCAN_CONSISTENCY_LEVEL_PROPERTY_DEFAULT = "ONE";

  public static final String DELETE_CONSISTENCY_LEVEL_PROPERTY = "cassandra.deleteconsistencylevel";
  public static final String DELETE_CONSISTENCY_LEVEL_PROPERTY_DEFAULT = "ONE";

[PDF] Link - Cassandra High Performance Cookbook¶

http://zopub.com/download?doc_id=519

저작자표시 비영리 변경금지

Posted by 사랑줍는거지

사랑줍는 거지