com.cloudera.oryx.app.als

Class PartitionedFeatureVectors

java.lang.Object

com.cloudera.oryx.app.als.PartitionedFeatureVectors

All Implemented Interfaces:

FeatureVectors

public final class PartitionedFeatureVectors
extends Object
implements FeatureVectors

Encapsulates a feature vectors, keyed by an ID, and associated data structures. This supports the representation of user-feature and item-feature vectors in ALSServingModel or ALSSpeedModel. This represents many partition of vectors; see FeatureVectorsPartition for a representation of one partition.

Constructor Summary

Constructors

Constructor and Description
PartitionedFeatureVectors(int numPartitions, ExecutorService executor)
PartitionedFeatureVectors(int numPartitions, ExecutorService executor, ToIntBiFunction<String,float[]> partitioner)

Method Summary

Modifier and Type	Method and Description
void	addAllIDsTo(Collection<String> allIDs) Adds all IDs that are mapped to a feature vector to a given collection
void	addAllRecentTo(Collection<String> allRecent) Add all recently set IDs to the given collection
float[]	getVector(String item)
double[]	getVTV(boolean background)
<T> Stream<T>	mapPartitionsParallel(Function<FeatureVectorsPartition,Stream<T>> fn, boolean background)
<T> Stream<T>	mapPartitionsParallel(Function<FeatureVectorsPartition,Stream<T>> fn, int[] candidateIndices, boolean background)
void	removeAllIDsFrom(Collection<String> allIDs) Removes all IDs that are mapped to a feature vector from a given collection
void	retainRecentAndIDs(Collection<String> newModelIDs) Given IDs that are part of a new model, and whose values are going to be sent later, retain only IDs that are also in the new model (or have been recently seen -- possibly since last model was built), and remove the rest.
void	setVector(String item, float[] vector) Sets the value of a feature vector for an ID.
int	size()
String	toString()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

PartitionedFeatureVectors

public PartitionedFeatureVectors(int numPartitions,
                                 ExecutorService executor)

PartitionedFeatureVectors

public PartitionedFeatureVectors(int numPartitions,
                                 ExecutorService executor,
                                 ToIntBiFunction<String,float[]> partitioner)

Method Detail

size

public int size()

Specified by:

size in interface FeatureVectors

Returns:

number of IDs / feature vectors

mapPartitionsParallel

public <T> Stream<T> mapPartitionsParallel(Function<FeatureVectorsPartition,Stream<T>> fn,
                                           boolean background)

mapPartitionsParallel

public <T> Stream<T> mapPartitionsParallel(Function<FeatureVectorsPartition,Stream<T>> fn,
                                           int[] candidateIndices,
                                           boolean background)

getVector

public float[] getVector(String item)

Specified by:

getVector in interface FeatureVectors

Parameters:

item - ID to get feature vector for

Returns:

feature vector for ID, or null if doesn't exist

setVector

public void setVector(String item,
                      float[] vector)

Description copied from interface: FeatureVectors

Sets the value of a feature vector for an ID. If no feature vector previously existed, it is considered a "recent" ID until the next FeatureVectors.retainRecentAndIDs(Collection) call.

Specified by:

setVector in interface FeatureVectors

Parameters:

item - ID to set feature vector for

vector - new feature vector

addAllIDsTo

public void addAllIDsTo(Collection<String> allIDs)

Description copied from interface: FeatureVectors

Adds all IDs that are mapped to a feature vector to a given collection

Specified by:

addAllIDsTo in interface FeatureVectors

Parameters:

allIDs - collection to add IDs to

removeAllIDsFrom

public void removeAllIDsFrom(Collection<String> allIDs)

Description copied from interface: FeatureVectors

Removes all IDs that are mapped to a feature vector from a given collection

Specified by:

removeAllIDsFrom in interface FeatureVectors

Parameters:

allIDs - collection to add IDs to

addAllRecentTo

public void addAllRecentTo(Collection<String> allRecent)

Description copied from interface: FeatureVectors

Add all recently set IDs to the given collection

Specified by:

addAllRecentTo in interface FeatureVectors

Parameters:

allRecent - collection to add IDs to

retainRecentAndIDs

public void retainRecentAndIDs(Collection<String> newModelIDs)

Description copied from interface: FeatureVectors

Given IDs that are part of a new model, and whose values are going to be sent later, retain only IDs that are also in the new model (or have been recently seen -- possibly since last model was built), and remove the rest.

Specified by:

retainRecentAndIDs in interface FeatureVectors

Parameters:

newModelIDs - new model IDs

getVTV

public double[] getVTV(boolean background)

Specified by:

getVTV in interface FeatureVectors

Parameters:

background - if true, a hint that this potentially expensive computation may be deprioritized and need not use all available resources to compute

Returns:

considering the feature vectors as the rows of a matrix V, this computes V^T * V, as a dense lower-triangular matrix, represented in BLAS 'packed' form, which is column-major.

toString

public String toString()

Overrides:

toString in class Object