Regularized Greedy Forest classifier
Description
Regularized Greedy Forest classifier
Regularized Greedy Forest classifier
Usage
# init <- RGF_Classifier$new(max_leaf = 1000, test_interval = 100,
# algorithm = "RGF", loss = "Log", reg_depth = 1.0,
# l2 = 0.1, sl2 = NULL, normalize = FALSE,
# min_samples_leaf = 10, n_iter = NULL,
# n_tree_search = 1, opt_interval = 100,
# learning_rate = 0.5, calc_prob = "sigmoid",
# n_jobs = 1, memory_policy = "generous",
# verbose = 0, init_model = NULL)
Details
the fit function builds a classifier from the training set (x, y).
the predict function predicts the class for x.
the predict_proba function predicts class probabilities for x.
the cleanup function removes tempfiles used by this model. See the issue https://github.com/RGF-team/rgf/issues/75, which explains in which cases the cleanup function applies.
the get_params function returns the parameters of the model.
the score function returns the mean accuracy on the given test data and labels.
the feature_importances function returns the feature importances for the data.
the dump_model function currently prints information about the fitted model in the console
the save_model function saves a model to a file from which training can do warm-start in the future.
Methods
RGF_Classifier$new(max_leaf = 1000, test_interval = 100,
algorithm = "RGF", loss = "Log", reg_depth = 1.0,
l2 = 0.1, sl2 = NULL, normalize = FALSE,
min_samples_leaf = 10, n_iter = NULL,
n_tree_search = 1, opt_interval = 100,
learning_rate = 0.5, calc_prob = "sigmoid",
n_jobs = 1, memory_policy = "generous",
verbose = 0, init_model = NULL)--------------fit(x, y, sample_weight = NULL)--------------predict(x)--------------predict_proba(x)--------------cleanup()--------------get_params(deep = TRUE)--------------score(x, y, sample_weight = NULL)--------------feature_importances()--------------dump_model()--------------save_model(filename)--------------
Super class
RGF::Internal_class -> RGF_Classifier
Methods
Public methods
Inherited methods
Method new()
Usage
RGF_Classifier$new(
max_leaf = 1000,
test_interval = 100,
algorithm = "RGF",
loss = "Log",
reg_depth = 1,
l2 = 0.1,
sl2 = NULL,
normalize = FALSE,
min_samples_leaf = 10,
n_iter = NULL,
n_tree_search = 1,
opt_interval = 100,
learning_rate = 0.5,
calc_prob = "sigmoid",
n_jobs = 1,
memory_policy = "generous",
verbose = 0,
init_model = NULL
)
Arguments
max_leaf-
an integer. Training will be terminated when the number of leaf nodes in the forest reaches this value.
test_interval-
an integer. Test interval in terms of the number of leaf nodes.
algorithm-
a character string specifying the Regularization algorithm. One of "RGF" (RGF with L2 regularization on leaf-only models), "RGF_Opt" (RGF with min-penalty regularization) or "RGF_Sib" (RGF with min-penalty regularization with the sum-to-zero sibling constraints).
loss-
a character string specifying the Loss function. One of "LS" (Square loss), "Expo" (Exponential loss) or "Log" (Logistic loss).
reg_depth-
a float. Must be no smaller than 1.0. Meant for being used with the algorithm RGF Opt or RGF Sib. A larger value penalizes deeper nodes more severely.
l2-
a float. Used to control the degree of L2 regularization.
sl2-
a float or NULL. Override L2 regularization parameter l2 for the process of growing the forest. That is, if specified, the weight correction process uses l2 and the forest growing process uses sl2. If NULL, no override takes place and l2 is used throughout training.
normalize-
a boolean. If True, training targets are normalized so that the average becomes zero.
min_samples_leaf-
an integer or a float. Minimum number of training data points in each leaf node. If an integer, then consider min_samples_leaf as the minimum number. If a float, then min_samples_leaf is a percentage and ceil(min_samples_leaf * n_samples) are the minimum number of samples for each node.
n_iter-
an integer or NULL. The number of iterations of coordinate descent to optimize weights. If NULL, 10 is used for loss = "LS" and 5 for loss = "Expo" or "Log".
n_tree_search-
an integer. The number of trees to be searched for the nodes to split. The most recently grown trees are searched first.
opt_interval-
an integer. Weight optimization interval in terms of the number of leaf nodes. For example, by default, weight optimization is performed every time approximately 100 leaf nodes are newly added to the forest.
learning_rate-
a float. Step size of Newton updates used in coordinate descent to optimize weights.
calc_prob-
a character string. One of "sigmoid" or "softmax". Method of probability calculation.
n_jobs-
an integer. The number of jobs (threads) to use for the computation. The substantial number of the jobs dependents on classes_ (The number of classes when fit is performed). If classes_ = 2, the substantial max number of the jobs is one. If classes_ > 2, the substantial max number of the jobs is the same as classes_. If n_jobs = 1, no parallel computing code is used at all regardless of classes_. If n_jobs = -1 and classes_ >= number of CPU, all CPUs are used. For n_jobs = -2, all CPUs but one are used. For n_jobs below -1, (n_cpus + 1 + n_jobs) are used.
memory_policy-
a character string. One of "conservative" (it uses less memory at the expense of longer runtime. Try only when with default value it uses too much memory) or "generous" (it runs faster using more memory by keeping the sorted orders of the features on memory for reuse). Memory using policy.
verbose-
an integer. Controls the verbosity of the tree building process.
init_model-
either NULL or a character string, optional (default=NULL). Filename of a previously saved model from which training should do warm-start. If model has been saved into multiple files, do not include numerical suffixes in the filename. NOTE: Make sure you haven't forgotten to increase the value of the max_leaf parameter regarding to the specified warm-start model because warm-start model trees are counted in the overall number of trees.
Method clone()
The objects of this class are cloneable with this method.
Usage
RGF_Classifier$clone(deep = FALSE)
Arguments
deep-
Whether to make a deep clone.
References
https://github.com/RGF-team/rgf/tree/master/python-package, Rie Johnson and Tong Zhang, Learning Nonlinear Functions Using Regularized Greedy Forest
Examples
try({
if (reticulate::py_available(initialize = FALSE)) {
if (reticulate::py_module_available("rgf.sklearn")) {
library(RGF)
set.seed(1)
x = matrix(runif(1000), nrow = 100, ncol = 10)
y = sample(1:2, 100, replace = TRUE)
RGF_class = RGF_Classifier$new(max_leaf = 50)
RGF_class$fit(x, y)
preds = RGF_class$predict_proba(x)
}
}
}, silent = TRUE)
try({
if (reticulate::py_available(initialize = FALSE)) {
if (reticulate::py_module_available("rgf.sklearn")) {
library(RGF)
set.seed(1)
x = matrix(runif(1000), nrow = 100, ncol = 10)
y = sample(1:2, 100, replace = TRUE)
RGF_class = RGF_Classifier$new(max_leaf = 50)
RGF_class$fit(x, y)
preds = RGF_class$predict_proba(x)
}
}
}, silent = TRUE)
, Ryosuke Fukatani
[cph] (Author of the python wrapper of the 'Regularized Greedy
Forest' machine learning algorithm), Nikita Titov [cph] (Author
of the python wrapper of the 'Regularized Greedy Forest'
machine learning algorithm), Tong Zhang [cph] (Author of the
'Regularized Greedy Forest' and of the Multi-core
implementation of Regularized Greedy Forest machine learning
algorithm), Rie Johnson [cph] (Author of the 'Regularized
Greedy Forest' machine learning algorithm)