IVFPQ index for the GPU. More...
#include <GpuIndexIVFPQ.h>
Public Member Functions | |
| GpuIndexIVFPQ (GpuResources *resources, int device, IndicesOptions indicesOptions, bool useFloat16LookupTables, const faiss::IndexIVFPQ *index) | |
| GpuIndexIVFPQ (GpuResources *resources, int device, int dims, int nlist, int subQuantizers, int bitsPerCode, bool usePrecomputed, IndicesOptions indicesOptions, bool useFloat16LookupTables, faiss::MetricType metric) | |
| Construct an empty index. | |
| void | copyFrom (const faiss::IndexIVFPQ *index) |
| void | copyTo (faiss::IndexIVFPQ *index) const |
| void | reserveMemory (size_t numVecs) |
| Reserve GPU memory in our inverted lists for this number of vectors. | |
| void | setPrecomputedCodes (bool enable) |
| Enable or disable pre-computed codes. | |
| bool | getPrecomputedCodes () const |
| Are pre-computed codes enabled? | |
| bool | getFloat16LookupTables () const |
| Are float16 residual distance lookup tables enabled? | |
| int | getNumSubQuantizers () const |
| Return the number of sub-quantizers we are using. | |
| int | getBitsPerCode () const |
| Return the number of bits per PQ code. | |
| int | getCentroidsPerSubQuantizer () const |
| Return the number of centroids per PQ code (2^bits per code) | |
| size_t | reclaimMemory () |
| void | reset () override |
| void | train (Index::idx_t n, const float *x) override |
| void | add_with_ids (Index::idx_t n, const float *x, const Index::idx_t *xids) override |
| void | search (faiss::Index::idx_t n, const float *x, faiss::Index::idx_t k, float *distances, faiss::Index::idx_t *labels) const override |
| void | set_typename () override |
| int | getListLength (int listId) const |
| std::vector< unsigned char > | getListCodes (int listId) const |
| std::vector< long > | getListIndices (int listId) const |
Public Member Functions inherited from faiss::gpu::GpuIndexIVF | |
| GpuIndexIVF (GpuResources *resources, int device, IndicesOptions indicesOptions, bool useFloat16CoarseQuantizer, int dims, faiss::MetricType metric, int nlist) | |
| GpuIndexIVF (GpuResources *resources, int device, IndicesOptions indicesOptions, int dims, faiss::MetricType metric, int nlist, GpuIndexFlat *quantizer) | |
| IndicesOptions | getIndicesOptions () const |
| What indices storage options are we using? | |
| bool | getUseFloat16CoarseQuantizer () const |
| Is our coarse quantizer storing and performing math in float16? | |
| void | copyFrom (const faiss::IndexIVF *index) |
| Copy what we need from the CPU equivalent. | |
| void | copyTo (faiss::IndexIVF *index) const |
| Copy what we have to the CPU equivalent. | |
| int | getNumLists () const |
| Returns the number of inverted lists we're managing. | |
| void | setNumProbes (int nprobe) |
| Sets the number of list probes per query. | |
| int | getNumProbes () const |
| Returns our current number of list probes per query. | |
| void | add (Index::idx_t n, const float *x) override |
Public Member Functions inherited from faiss::gpu::GpuIndex | |
| GpuIndex (GpuResources *resources, int device, int dims, faiss::MetricType metric) | |
| int | getDevice () const |
| GpuResources * | getResources () |
Public Member Functions inherited from faiss::Index | |
| Index (idx_t d=0, MetricType metric=METRIC_INNER_PRODUCT) | |
| virtual void | range_search (idx_t n, const float *x, float radius, RangeSearchResult *result) const |
| void | assign (idx_t n, const float *x, idx_t *labels, idx_t k=1) |
| virtual long | remove_ids (const IDSelector &sel) |
| virtual void | reconstruct (idx_t key, float *recons) const |
| virtual void | reconstruct_n (idx_t i0, idx_t ni, float *recons) const |
| void | compute_residual (const float *x, float *residual, idx_t key) const |
| void | display () const |
| virtual std::string | get_typename () const |
Additional Inherited Members | |
Public Types inherited from faiss::Index | |
| typedef long | idx_t |
| all indices are this type | |
Public Attributes inherited from faiss::Index | |
| std::string | index_typename |
| int | d |
| vector dimension | |
| idx_t | ntotal |
| total nb of indexed vectors | |
| bool | verbose |
| verbosity level | |
| bool | is_trained |
| set if the Index does not require training, or if training is done already | |
| MetricType | metric_type |
| type of metric this index uses for search | |
Protected Member Functions inherited from faiss::gpu::GpuIndexIVF | |
| void | trainQuantizer_ (faiss::Index::idx_t n, const float *x) |
Protected Attributes inherited from faiss::gpu::GpuIndexIVF | |
| const IndicesOptions | indicesOptions_ |
| How should indices be stored on the GPU? | |
| const bool | useFloat16CoarseQuantizer_ |
| int | nlist_ |
| Number of inverted lists that we manage. | |
| int | nprobe_ |
| Number of inverted list probes per query. | |
| ClusteringParameters | cp_ |
| Ability to override default clustering parameters. | |
| GpuIndexFlat * | quantizer_ |
| Quantizer for inverted lists. | |
| bool | ownsQuantizer_ |
| Do we own the above quantizer instance? | |
Protected Attributes inherited from faiss::gpu::GpuIndex | |
| GpuResources * | resources_ |
| Manages streans, cuBLAS handles and scratch memory for devices. | |
| int | device_ |
| The GPU device we are resident on. | |
IVFPQ index for the GPU.
Definition at line 25 of file GpuIndexIVFPQ.h.
| faiss::gpu::GpuIndexIVFPQ::GpuIndexIVFPQ | ( | GpuResources * | resources, |
| int | device, | ||
| IndicesOptions | indicesOptions, | ||
| bool | useFloat16LookupTables, | ||
| const faiss::IndexIVFPQ * | index | ||
| ) |
Construct from a pre-existing faiss::IndexIVFPQ instance, copying data over to the given GPU, if the input index is trained.
Definition at line 26 of file GpuIndexIVFPQ.cu.
|
overridevirtual |
x and xids can be resident on the CPU or any GPU; the proper copies are performed
Reimplemented from faiss::Index.
Definition at line 326 of file GpuIndexIVFPQ.cu.
| void faiss::gpu::GpuIndexIVFPQ::copyFrom | ( | const faiss::IndexIVFPQ * | index | ) |
Reserve space on the GPU for the inverted lists for num vectors, assumed equally distributed among Initialize ourselves from the given CPU index; will overwrite all data in ourselves
Definition at line 92 of file GpuIndexIVFPQ.cu.
| void faiss::gpu::GpuIndexIVFPQ::copyTo | ( | faiss::IndexIVFPQ * | index | ) | const |
Copy ourselves to the given CPU index; will overwrite all data in the index instance
Definition at line 149 of file GpuIndexIVFPQ.cu.
| std::vector< unsigned char > faiss::gpu::GpuIndexIVFPQ::getListCodes | ( | int | listId | ) | const |
For debugging purposes, return the list codes of a particular list
Definition at line 420 of file GpuIndexIVFPQ.cu.
| std::vector< long > faiss::gpu::GpuIndexIVFPQ::getListIndices | ( | int | listId | ) | const |
For debugging purposes, return the list indices of a particular list
Definition at line 428 of file GpuIndexIVFPQ.cu.
| int faiss::gpu::GpuIndexIVFPQ::getListLength | ( | int | listId | ) | const |
For debugging purposes, return the list length of a particular list
Definition at line 414 of file GpuIndexIVFPQ.cu.
| size_t faiss::gpu::GpuIndexIVFPQ::reclaimMemory | ( | ) |
After adding vectors, one can call this to reclaim device memory to exactly the amount needed. Returns space reclaimed in bytes
Definition at line 242 of file GpuIndexIVFPQ.cu.
|
overridevirtual |
Clears out all inverted lists, but retains the coarse and product centroid information
Implements faiss::Index.
Definition at line 252 of file GpuIndexIVFPQ.cu.
|
overridevirtual |
x, distances and labels can be resident on the CPU or any GPU; copies are performed as needed
Implements faiss::Index.
Definition at line 359 of file GpuIndexIVFPQ.cu.
|
overridevirtual |
Perform training on a representative set of vectors
| n | nb of training vectors |
| x | training vecors, size n * d |
Reimplemented from faiss::Index.
Definition at line 307 of file GpuIndexIVFPQ.cu.
1.8.5