Welcome to the Kerch package

This package is meant for various kernel-based operations. For the moment, only the kernel module is available, but a lot of models will follow soon. If you experience any bug, please let me now: this is only a pre-alpha version.

Install

As for now, there are two ways to install the package.

Using PIP

Using pip, it suffices to run pip install kerch. Just rerun this command to update the package to its newest version.

From source

You can also install the package directly from the GitHub repository.

git clone --recursive https://github.com/hdeplaen/kerch
cd kerch
python setup.py install

Kernel Module

Introduction

This module contains many different types of kernels. Each kernel is created based on some hyperparameters and a sample dataset.

If no sample dataset is provided, a random one will be initialized. This dataset can always be reinitialized (init_sample) or the alue of the datapoints can be updated updated (update_sample). In the latter case, the dimensions have to be matching. Furthermore, the sample dataset can also work in a stochastic manner, of which the indices can be controlled through the reset method.

Both the value of the sample datapoints as the hyperparameters are compatible with gradient graphs of PyTorch. If such a graph is to be computed, this has to be specifically specified during constructions.

All kernels can be centered, either implicitly using statistics on the kernel matrix of the sample dataset, either explicitly using a statistic on the explicit feature map. In the former case, this cannot be extended to fully out-of-sample computations.

At last, a Nystrom kernel is also implemented, which created an explicit feature map based on any kernel (possibly implicit), using eigendocomposition. Among other things, this can serve as a solution for centering fully out-of-sample kernel matrices of implicitly defined kernels.

The general structure of the module is based around an abstract kernel class base, of which kerch.kernle.implicit and explicit inherit. All other kernels inherit of one of these two at the exception of polynomial which directly inherits base as it has a primal formulation and a dual formulation which can be computed otherwise than with an inner product of the explicit feature map.

Kernel Factory

kerch.kernel.factory(type='rbf', **kwargs) → _Projected

Creates a kernel based on the specified name with the specified arguments. This is the same as calling kerch.kernel.name(**kwargs) (if name is not a string here). This allows for the creation of kernel where the name of kernel is passed as a string.

Parameters:

type (str, optional) – Type of kernel chosen. For the possible choices, please refer to the (non-abstract) classes herebelow., defaults to rbf
**kwargs (dict, optional) – Arguments to be passed to the kernel constructor, such as sample or sigma. If an argument is passed that does not exist (e.g. sigma to a linear kernel), it will just be neglected. For the default values, please refer to the default values of the requested kernel.

Returns:

An instantiation of the specified kernel.

Examples

import kerch
import numpy as np
from matplotlib import pyplot as plt

sample = np.sin(np.arange(0,15) / np.pi) + .1
oos = np.sin(np.arange(15,30) / np.pi) + .1

k = kerch.kernel.factory(type="polynomial", sample=sample, center=True, normalize=True)

fig, axs = plt.subplots(2,2)

axs[0,0].imshow(k.K, vmin=-1, vmax=1)
axs[0,0].set_title("Sample - Sample")

axs[0,1].imshow(k.k(y=oos), vmin=-1, vmax=1)
axs[0,1].set_title("Sample - OOS")

axs[1,0].imshow(k.k(x=oos), vmin=-1, vmax=1)
axs[1,0].set_title("OOS - Sample")

im = axs[1,1].imshow(k.k(x=oos, y=oos), vmin=-1, vmax=1)
axs[1,1].set_title("OOS - OOS")

for ax in axs.flat:
    ax.set_xticks([])
    ax.set_yticks([])

fig.colorbar(im, ax=axs.ravel().tolist())

(Source code, png, hires.png, pdf)

Different Kernels

Generic Kernels

Linear Kernel

Class

kerch.kernel.linear: alias of <module ‘kerch.kernel.linear’ from ‘/home/docs/checkouts/readthedocs.org/user_builds/kerch/checkouts/latest/kerch/kernel/linear.py’>

Examples

Sine

import kerch
import numpy as np
from matplotlib import pyplot as plt

x = np.sin(np.arange(50) / np.pi) + 1.5
plt.figure(0)
plt.plot(x)

k = kerch.kernel.linear(sample=x)
plt.figure(1)
plt.imshow(k.K)
plt.colorbar()

(Source code)

RBF Kernel

Class

kerch.kernel.rbf: alias of <module ‘kerch.kernel.rbf’ from ‘/home/docs/checkouts/readthedocs.org/user_builds/kerch/checkouts/latest/kerch/kernel/rbf.py’>

Examples

Sine

import kerch
import numpy as np
from matplotlib import pyplot as plt

x = np.sin(np.arange(50) / np.pi)
plt.figure(0)
plt.plot(x)

k = kerch.kernel.rbf(sample=x)

plt.figure(1)
plt.imshow(k.K)
plt.colorbar()
plt.title("Sigma = "+str(k.sigma))

k.sigma = 1

plt.figure(2)
plt.imshow(k.K)
plt.colorbar()
plt.title("Sigma = "+str(k.sigma))

(Source code)

Time

import kerch
from matplotlib import pyplot as plt

k = kerch.kernel.rbf(sample=range(10), sigma=3)

plt.imshow(k.K)
plt.colorbar()
plt.title("RBF with sigma " + str(k.sigma))

(Source code)

Laplacian Kernel

kerch.kernel.laplacian: alias of <module ‘kerch.kernel.laplacian’ from ‘/home/docs/checkouts/readthedocs.org/user_builds/kerch/checkouts/latest/kerch/kernel/laplacian.py’>

Polynomial Kernel

Class

kerch.kernel.polynomial: alias of <module ‘kerch.kernel.polynomial’ from ‘/home/docs/checkouts/readthedocs.org/user_builds/kerch/checkouts/latest/kerch/kernel/polynomial.py’>

Examples

Sine

import kerch
import numpy as np
from matplotlib import pyplot as plt

x = np.sin(np.arange(50) / np.pi)
plt.figure(0)
plt.plot(x)

k = kerch.kernel.polynomial(sample=x, degree=2)
plt.figure(1)
plt.imshow(k.K)
plt.colorbar()

(Source code)

Cosine Kernel

Class

kerch.kernel.cosine: alias of <module ‘kerch.kernel.cosine’ from ‘/home/docs/checkouts/readthedocs.org/user_builds/kerch/checkouts/latest/kerch/kernel/cosine.py’>

Examples

Sine

import kerch
import numpy as np
from matplotlib import pyplot as plt

x = np.sin(np.arange(50) / np.pi) + 1.5
plt.figure(0)
plt.plot(x)

k = kerch.kernel.cosine(sample=x, center=True)
plt.figure(1)
plt.imshow(k.K)
plt.colorbar()

(Source code)

Sigmoid Kernel

kerch.kernel.sigmoid: alias of <module ‘kerch.kernel.sigmoid’ from ‘/home/docs/checkouts/readthedocs.org/user_builds/kerch/checkouts/latest/kerch/kernel/sigmoid.py’>

Random Fourier Features Kernel

Class

kerch.kernel.rff: alias of <module ‘kerch.kernel.rff’ from ‘/home/docs/checkouts/readthedocs.org/user_builds/kerch/checkouts/latest/kerch/kernel/rff.py’>

Examples

Sine

import kerch
import numpy as np
from matplotlib import pyplot as plt

x = np.sin(np.arange(50) / np.pi)

k_rbf = kerch.kernel.rbf(sample=x, sigma=1)
k_rff = kerch.kernel.rff(sample=x, num_weights=50)

fig, axs = plt.subplots(1, 2)

axs[0].imshow(k_rbf.K)
axs[0].set_title("RBF")

im = axs[1].imshow(k_rff.K)
axs[1].set_title("RFF")

fig.colorbar(im, ax=axs.ravel().tolist(), orientation='horizontal')

(Source code)

Nystrom Kernel

kerch.kernel.nystrom: alias of <module ‘kerch.kernel.nystrom’ from ‘/home/docs/checkouts/readthedocs.org/user_builds/kerch/checkouts/latest/kerch/kernel/nystrom.py’>

Network-based kernels

Explicit Network-based Kernel

kerch.kernel.explicit_nn: alias of <module ‘kerch.kernel.explicit_nn’ from ‘/home/docs/checkouts/readthedocs.org/user_builds/kerch/checkouts/latest/kerch/kernel/explicit_nn.py’>

Implicit Network-based Kernel

kerch.kernel.implicit_nn: alias of <module ‘kerch.kernel.implicit_nn’ from ‘/home/docs/checkouts/readthedocs.org/user_builds/kerch/checkouts/latest/kerch/kernel/implicit_nn.py’>

Time Kernels

The idea behind time kernels is that time has the same local effect at all time, or in other words that the kernels are translational invariant. We typically consider the following kernels:

Indicator Kernel

Class

kerch.kernel.indicator: alias of <module ‘kerch.kernel.indicator’ from ‘/home/docs/checkouts/readthedocs.org/user_builds/kerch/checkouts/latest/kerch/kernel/indicator.py’>

Examples

Linear (Time)

import kerch
from matplotlib import pyplot as plt

k = kerch.kernel.indicator(sample=range(10), lag=3)
plt.imshow(k.K)
plt.colorbar()
plt.title("Indicator with lag " + str(k.lag))

(Source code)

Hat Kernel

Class

kerch.kernel.hat: alias of <module ‘kerch.kernel.hat’ from ‘/home/docs/checkouts/readthedocs.org/user_builds/kerch/checkouts/latest/kerch/kernel/hat.py’>

Examples

Linear (Time)

import kerch
from matplotlib import pyplot as plt

k = kerch.kernel.hat(sample=range(10), lag=3)
plt.imshow(k.K)
plt.colorbar()
plt.title("Hat with lag " + str(k.lag))

(Source code)

Vision Kernels

Additive Chi Squared Kernel

kerch.kernel.additive_chi2: alias of <module ‘kerch.kernel.additive_chi2’ from ‘/home/docs/checkouts/readthedocs.org/user_builds/kerch/checkouts/latest/kerch/kernel/additive_chi2.py’>

Skewed Chi Squared Kernel

kerch.kernel.skewed_chi2: alias of <module ‘kerch.kernel.skewed_chi2’ from ‘/home/docs/checkouts/readthedocs.org/user_builds/kerch/checkouts/latest/kerch/kernel/skewed_chi2.py’>

Abstract Kernels

Exponential Kernel

Explicit Kernel

Implicit Kernel

Base Kernel

Examples

Some examples of what is possible. Please refer to the rest of the documentation for more examples.

Training and tuning an LS-SVM

import kerch

tr_set, _, _, _ = kerch.dataset.factory("two_moons",   # which dataset
                                     tr_size=250)      # training size
mdl = kerch.model.LSSVM(type="rbf",                    # kernel type
                     representation="dual")            # initiate model
mdl.set_data_prop(data=tr_set[0],                      # data
                  labels=tr_set[1],                    # corresponding labels
                  proportions=[1, 0, 0])               # initiate dataset
mdl.hyperopt({"gamma", "sigma"},                       # define which parameters to tune
             max_evals=500,                            # define how many trials
             k=10)                                     # 10-fold cross-validation
mdl.fit()                                              # fit the optimal parameters found
kerch.plot.plot_model(mdl)                             # plot the model using the built-in method

(Source code)

Out-of-sample normalized and centered kernels

import kerch
import numpy as np
from matplotlib import pyplot as plt

sample = np.sin(np.arange(0,15) / np.pi) + .1
oos = np.sin(np.arange(15,30) / np.pi) + .1

k = kerch.kernel.factory(type="polynomial", sample=sample, center=True, normalize=True)

fig, axs = plt.subplots(2,2)

axs[0,0].imshow(k.K, vmin=-1, vmax=1)
axs[0,0].set_title("Sample -Sample")

axs[0,1].imshow(k.k(y=oos), vmin=-1, vmax=1)
axs[0,1].set_title("Sample - OOS")

axs[1,0].imshow(k.k(x=oos), vmin=-1, vmax=1)
axs[1,0].set_title("OOS - Sample")

im = axs[1,1].imshow(k.k(x=oos, y=oos), vmin=-1, vmax=1)
axs[1,1].set_title("OOS - OOS")

for ax in axs.flat:
    ax.set_xticks([])
    ax.set_yticks([])

fig.colorbar(im, ax=axs.ravel().tolist())

(Source code, png, hires.png, pdf)

Indices and tables

Index

Module Index

Search Page