com.cloudera.oryx.example.speed

Class ExampleSpeedModelManager

java.lang.Object

com.cloudera.oryx.api.speed.AbstractSpeedModelManager<String,String,String>

com.cloudera.oryx.example.speed.ExampleSpeedModelManager

All Implemented Interfaces:

SpeedModelManager<String,String,String>, Closeable, AutoCloseable

public final class ExampleSpeedModelManager
extends AbstractSpeedModelManager<String,String,String>

Also counts and emits counts of number of distinct words that occur with words. Listens for updates from the Batch Layer, which give the current correct count at its last run. Updates these counts approximately in response to the same data stream that the Batch Layer sees, but assumes all words seen are new and distinct, which is only approximately true. Emits updates of the form "word,count".

Constructor Summary

Constructors

Constructor and Description
ExampleSpeedModelManager()

Method Summary

Modifier and Type	Method and Description
Iterable<String>	buildUpdates(org.apache.spark.api.java.JavaPairRDD<String,String> newData)
void	consumeKeyMessage(String key, String message, org.apache.hadoop.conf.Configuration hadoopConf) Convenience method that is called by the default implementation of AbstractSpeedModelManager.consume(Iterator, Configuration), to process one key-message pair.

Methods inherited from class com.cloudera.oryx.api.speed.AbstractSpeedModelManager

consume

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface com.cloudera.oryx.api.speed.SpeedModelManager

close

Constructor Detail

ExampleSpeedModelManager

public ExampleSpeedModelManager()

Method Detail

consumeKeyMessage

public void consumeKeyMessage(String key,
                              String message,
                              org.apache.hadoop.conf.Configuration hadoopConf)
                       throws IOException

Description copied from class: AbstractSpeedModelManager

Convenience method that is called by the default implementation of AbstractSpeedModelManager.consume(Iterator, Configuration), to process one key-message pair. It does nothing, except log the message. This should generally be overridden if and only if AbstractSpeedModelManager.consume(Iterator, Configuration) is not.

Overrides:

consumeKeyMessage in class AbstractSpeedModelManager<String,String,String>

Parameters:

key - key to process (non-null)

message - message to process

hadoopConf - Hadoop configuration for process

Throws:

IOException - if an error occurs while processing the message

buildUpdates

public Iterable<String> buildUpdates(org.apache.spark.api.java.JavaPairRDD<String,String> newData)

Parameters:

newData - RDD of raw new data from the topic

Returns:

updates to publish on the update topic