com.netflix.nicobar.cassandra

Class CassandraArchiveRepository

java.lang.Object

com.netflix.nicobar.cassandra.CassandraArchiveRepository

All Implemented Interfaces:

ArchiveRepository

public class CassandraArchiveRepository
extends java.lang.Object
implements ArchiveRepository

Data access object of ScriptArchives stored in Cassandra. This implementation is based on the Astyanax and requires CQL 3 support to be enabled.

The query algorithm attempts to divide up read operations such that they won't overwhelm Cassandra if many instances are using this implementation to poll for updates. Upon insertion, all archives are assigned a shard number calculated as (moduleId.hashCode() % shardNum). The shard number is subsequently inserted into a column for which a secondary index has been defined. RepositoryView poller methods will first search each shard for any rows with an update timestamp greater than the last poll time, and if any are found, the contents of those archives are loaded in small batches.

 Default Schema:

 CREATE TABLE script_repo (
    module_id varchar,
    module_name varchar,
    module_version varchar,
    shard_num int,
    last_update timestamp,
    module_spec varchar,
    archive_content_hash blob,
    archive_content blob,
 PRIMARY KEY (module_id)
 );

 CREATE INDEX script_repo_shard_num_index on script_repo (shard_num);
 

See CassandraArchiveRepositoryConfig to override the default table name.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	CassandraArchiveRepository.Columns column names
protected class	CassandraArchiveRepository.DefaultView

Field Summary

Fields

Modifier and Type	Field and Description
protected RepositoryView	defaultView

Constructor Summary

Constructors

Constructor and Description
CassandraArchiveRepository(CassandraArchiveRepositoryConfig config) Construct a instance of the repository with the given configuration
CassandraArchiveRepository(CassandraArchiveRepositoryConfig config, RepositoryView defaultView) Construct a instance of the repository with the given configuration

Method Summary

Methods

Modifier and Type	Method and Description
protected byte[]	calculateHash(byte[] content)
protected int	calculateShardNum(ModuleId moduleId)
void	deleteArchive(ModuleId moduleId) Delete an archive by ID
protected java.lang.String	generateSelectByShardCql(java.util.EnumSet<?> columns, java.lang.Integer shardNum) Generate the CQL to select specific columns by shard number.
CassandraArchiveRepositoryConfig	getConfig()
RepositoryView	getDefaultView() The default view reports all archives inserted into this repository.
java.lang.String	getRepositoryId() Get the ID of this repository
protected java.lang.Iterable<com.netflix.astyanax.model.Row<java.lang.String,java.lang.String>>	getRows(java.util.EnumSet<?> columns) Get all of the rows in in the table.
java.util.Set<ScriptArchive>	getScriptArchives(java.util.Set<ModuleId> moduleIds) Get all of the ScriptArchives for the given set of moduleIds.
RepositoryView	getView(java.lang.String view) No named views supported by this repository! Throws UnsupportedOperationException.
void	insertArchive(JarScriptArchive jarScriptArchive) insert a Jar into the script archive
void	insertArchive(JarScriptArchive jarScriptArchive, java.util.Map<java.lang.String,java.lang.Object> initialDeploySpecs) Unsupported.
protected boolean	verifyHash(byte[] expectedHashCode, byte[] content)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

defaultView

protected final RepositoryView defaultView

Constructor Detail

CassandraArchiveRepository

public CassandraArchiveRepository(CassandraArchiveRepositoryConfig config)

Construct a instance of the repository with the given configuration

Parameters:

config -

CassandraArchiveRepository

public CassandraArchiveRepository(CassandraArchiveRepositoryConfig config,
                          RepositoryView defaultView)

Construct a instance of the repository with the given configuration

Parameters:

config -

Method Detail

getRepositoryId

public java.lang.String getRepositoryId()

Description copied from interface: ArchiveRepository

Get the ID of this repository

Specified by:

getRepositoryId in interface ArchiveRepository

Returns:

the id string.

getDefaultView

public RepositoryView getDefaultView()

The default view reports all archives inserted into this repository.

Specified by:

getDefaultView in interface ArchiveRepository

Returns:

the default view into all archives.

getView

public RepositoryView getView(java.lang.String view)

No named views supported by this repository! Throws UnsupportedOperationException.

Specified by:

getView in interface ArchiveRepository

Parameters:

view - the name of the view.

Returns:

a RepositoryView that matches the given name or null if one wasn't found.

insertArchive

public void insertArchive(JarScriptArchive jarScriptArchive)
                   throws java.io.IOException

insert a Jar into the script archive

Specified by:

insertArchive in interface ArchiveRepository

Parameters:

jarScriptArchive - script archive which describes the jar and the ModuleSpec which should be inserted

Throws:

java.io.IOException

insertArchive

public void insertArchive(JarScriptArchive jarScriptArchive,
                 java.util.Map<java.lang.String,java.lang.Object> initialDeploySpecs)
                   throws java.io.IOException

Unsupported.

Specified by:

insertArchive in interface ArchiveRepository

Parameters:

jarScriptArchive - script archive which describes the jar and the ModuleSpec which should be inserted

initialDeploySpecs - a set of initial deployment specs.

Throws:

java.io.IOException

getScriptArchives

public java.util.Set<ScriptArchive> getScriptArchives(java.util.Set<ModuleId> moduleIds)
                                               throws java.io.IOException

Get all of the ScriptArchives for the given set of moduleIds. Will perform the operation in batches as specified by CassandraArchiveRepositoryConfig.getArchiveFetchBatchSize() and outputs the jar files in the path specified by CassandraArchiveRepositoryConfig.getArchiveOutputDirectory().

Specified by:

getScriptArchives in interface ArchiveRepository

Parameters:

moduleIds - keys to search for

Returns:

set of ScriptArchives retrieved from the database

Throws:

java.io.IOException

deleteArchive

public void deleteArchive(ModuleId moduleId)
                   throws java.io.IOException

Delete an archive by ID

Specified by:

deleteArchive in interface ArchiveRepository

Parameters:

moduleId - module id to delete

Throws:

java.io.IOException

getRows

protected java.lang.Iterable<com.netflix.astyanax.model.Row<java.lang.String,java.lang.String>> getRows(java.util.EnumSet<?> columns)
                                                                                                 throws java.lang.Exception

Get all of the rows in in the table. Attempts to reduce the load on cassandra by splitting up the query into smaller sub-queries

Parameters:

columns - which columns to select

Returns:

result rows

Throws:

java.lang.Exception

generateSelectByShardCql

protected java.lang.String generateSelectByShardCql(java.util.EnumSet<?> columns,
                                        java.lang.Integer shardNum)

Generate the CQL to select specific columns by shard number.

      SELECT ${columns}... FROM script_repo WHERE shard_num = ?
 

verifyHash

protected boolean verifyHash(byte[] expectedHashCode,
                 byte[] content)

calculateHash

protected byte[] calculateHash(byte[] content)

calculateShardNum

protected int calculateShardNum(ModuleId moduleId)

getConfig

public CassandraArchiveRepositoryConfig getConfig()

Returns:

configuration settings for this repository