scpanel.SVMRFECV ================ .. py:module:: scpanel.SVMRFECV .. autoapi-nested-parse:: Recursive feature elimination for feature ranking Classes ------- .. autoapisummary:: scpanel.SVMRFECV.RFE scpanel.SVMRFECV.RFECV Functions --------- .. autoapisummary:: scpanel.SVMRFECV._rfe_single_fit Module Contents --------------- .. py:function:: _rfe_single_fit(rfe, estimator, X, y, train_idx, val_idx, scorer, sample_weight=None) Return the score for a fit across one fold. .. py:class:: RFE(estimator: sklearn.svm._classes.SVC, *, n_features_to_select=None, step=1, verbose=0, importance_getter='auto') Bases: :py:obj:`sklearn.feature_selection._base.SelectorMixin`, :py:obj:`sklearn.base.MetaEstimatorMixin`, :py:obj:`sklearn.base.BaseEstimator` Feature ranking with recursive feature elimination. Given an external estimator that assigns weights to features (e.g., the coefficients of a linear model), the goal of recursive feature elimination (RFE) is to select features by recursively considering smaller and smaller sets of features. First, the estimator is trained on the initial set of features and the importance of each feature is obtained either through any specific attribute or callable. Then, the least important features are pruned from current set of features. That procedure is recursively repeated on the pruned set until the desired number of features to select is eventually reached. Read more in the :ref:`User Guide `. :param estimator: A supervised learning estimator with a ``fit`` method that provides information about feature importance (e.g. `coef_`, `feature_importances_`). :type estimator: ``Estimator`` instance :param n_features_to_select: The number of features to select. If `None`, half of the features are selected. If integer, the parameter is the absolute number of features to select. If float between 0 and 1, it is the fraction of features to select. .. versionchanged:: 0.24 Added float values for fractions. :type n_features_to_select: int or float, default=None :param step: If greater than or equal to 1, then ``step`` corresponds to the (integer) number of features to remove at each iteration. If within (0.0, 1.0), then ``step`` corresponds to the percentage (rounded down) of features to remove at each iteration. :type step: int or float, default=1 :param verbose: Controls verbosity of output. :type verbose: int, default=0 :param importance_getter: If 'auto', uses the feature importance either through a `coef_` or `feature_importances_` attributes of estimator. Also accepts a string that specifies an attribute name/path for extracting feature importance (implemented with `attrgetter`). For example, give `regressor_.coef_` in case of :class:`~sklearn.compose.TransformedTargetRegressor` or `named_steps.clf.feature_importances_` in case of class:`~sklearn.pipeline.Pipeline` with its last step named `clf`. If `callable`, overrides the default feature importance getter. The callable is passed with the fitted estimator and it should return importance for each feature. .. versionadded:: 0.24 :type importance_getter: str or callable, default='auto' .. attribute:: classes_ The classes labels. Only available when `estimator` is a classifier. :type: ndarray of shape (n_classes,) .. attribute:: estimator_ The fitted estimator used to select features. :type: ``Estimator`` instance .. attribute:: n_features_ The number of selected features. :type: int .. attribute:: n_features_in_ Number of features seen during :term:`fit`. Only defined if the underlying estimator exposes such an attribute when fit. .. versionadded:: 0.24 :type: int .. attribute:: feature_names_in_ Names of features seen during :term:`fit`. Defined only when `X` has feature names that are all strings. .. versionadded:: 1.0 :type: ndarray of shape (`n_features_in_`,) .. attribute:: ranking_ The feature ranking, such that ``ranking_[i]`` corresponds to the ranking position of the i-th feature. Selected (i.e., estimated best) features are assigned rank 1. :type: ndarray of shape (n_features,) .. attribute:: support_ The mask of selected features. :type: ndarray of shape (n_features,) .. seealso:: :obj:`RFECV` Recursive feature elimination with built-in cross-validated selection of the best number of features. :obj:`SelectFromModel` Feature selection based on thresholds of importance weights. :obj:`SequentialFeatureSelector` Sequential cross-validation based feature selection. Does not rely on importance weights. .. rubric:: Notes Allows NaN/Inf in the input if the underlying estimator does as well. .. rubric:: References .. [1] Guyon, I., Weston, J., Barnhill, S., & Vapnik, V., "Gene selection for cancer classification using support vector machines", Mach. Learn., 46(1-3), 389--422, 2002. .. rubric:: Examples The following example shows how to retrieve the 5 most informative features in the Friedman #1 dataset. >>> from sklearn.datasets import make_friedman1 >>> from sklearn.feature_selection import RFE >>> from sklearn.svm import SVR >>> X, y = make_friedman1(n_samples=50, n_features=10, random_state=0) >>> estimator = SVR(kernel="linear") >>> selector = RFE(estimator, n_features_to_select=5, step=1) >>> selector = selector.fit(X, y) >>> selector.support_ array([ True, True, True, True, True, False, False, False, False, False]) >>> selector.ranking_ array([1, 1, 1, 1, 1, 6, 4, 3, 2, 5]) .. py:attribute:: estimator .. py:attribute:: n_features_to_select .. py:attribute:: step .. py:attribute:: importance_getter .. py:attribute:: verbose .. py:property:: _estimator_type .. py:property:: classes_ Classes labels available when `estimator` is a classifier. :rtype: ndarray of shape (n_classes,) .. py:method:: fit(X: numpy.ndarray, y: numpy.ndarray, **fit_params) -> RFE Fit the RFE model and then the underlying estimator on the selected features. :param X: The training input samples. :type X: {array-like, sparse matrix} of shape (n_samples, n_features) :param y: The target values. :type y: array-like of shape (n_samples,) :param \*\*fit_params: Additional parameters passed to the `fit` method of the underlying estimator. :type \*\*fit_params: dict :returns: **self** -- Fitted estimator. :rtype: object .. py:method:: _fit(X: numpy.ndarray, y: numpy.ndarray, step_score: None = None, **fit_params) -> RFE .. py:method:: predict(X) Reduce X to the selected features and then predict using the underlying estimator. :param X: The input samples. :type X: array of shape [n_samples, n_features] :returns: **y** -- The predicted target values. :rtype: array of shape [n_samples] .. py:method:: score(X, y, **fit_params) Reduce X to the selected features and return the score of the underlying estimator. :param X: The input samples. :type X: array of shape [n_samples, n_features] :param y: The target values. :type y: array of shape [n_samples] :param \*\*fit_params: Parameters to pass to the `score` method of the underlying estimator. .. versionadded:: 1.0 :type \*\*fit_params: dict :returns: **score** -- Score of the underlying base estimator computed with the selected features returned by `rfe.transform(X)` and `y`. :rtype: float .. py:method:: _get_support_mask() .. py:method:: decision_function(X) Compute the decision function of ``X``. :param X: The input samples. Internally, it will be converted to ``dtype=np.float32`` and if a sparse matrix is provided to a sparse ``csr_matrix``. :type X: {array-like or sparse matrix} of shape (n_samples, n_features) :returns: **score** -- The decision function of the input samples. The order of the classes corresponds to that in the attribute :term:`classes_`. Regression and binary classification produce an array of shape [n_samples]. :rtype: array, shape = [n_samples, n_classes] or [n_samples] .. py:method:: predict_proba(X) Predict class probabilities for X. :param X: The input samples. Internally, it will be converted to ``dtype=np.float32`` and if a sparse matrix is provided to a sparse ``csr_matrix``. :type X: {array-like or sparse matrix} of shape (n_samples, n_features) :returns: **p** -- The class probabilities of the input samples. The order of the classes corresponds to that in the attribute :term:`classes_`. :rtype: array of shape (n_samples, n_classes) .. py:method:: predict_log_proba(X) Predict class log-probabilities for X. :param X: The input samples. :type X: array of shape [n_samples, n_features] :returns: **p** -- The class log-probabilities of the input samples. The order of the classes corresponds to that in the attribute :term:`classes_`. :rtype: array of shape (n_samples, n_classes) .. py:method:: _more_tags() -> Dict[str, bool] .. py:class:: RFECV(estimator: sklearn.svm._classes.SVC, *, step=1, min_features_to_select=1, cv=None, scoring=None, verbose=0, n_jobs=None, importance_getter='auto') Bases: :py:obj:`RFE` Recursive feature elimination with cross-validation to select the number of features. See glossary entry for :term:`cross-validation estimator`. Read more in the :ref:`User Guide `. :param estimator: A supervised learning estimator with a ``fit`` method that provides information about feature importance either through a ``coef_`` attribute or through a ``feature_importances_`` attribute. :type estimator: ``Estimator`` instance :param step: If greater than or equal to 1, then ``step`` corresponds to the (integer) number of features to remove at each iteration. If within (0.0, 1.0), then ``step`` corresponds to the percentage (rounded down) of features to remove at each iteration. Note that the last iteration may remove fewer than ``step`` features in order to reach ``min_features_to_select``. :type step: int or float, default=1 :param min_features_to_select: The minimum number of features to be selected. This number of features will always be scored, even if the difference between the original feature count and ``min_features_to_select`` isn't divisible by ``step``. .. versionadded:: 0.20 :type min_features_to_select: int, default=1 :param cv: Determines the cross-validation splitting strategy. Possible inputs for cv are: - None, to use the default 5-fold cross-validation, - integer, to specify the number of folds. - :term:`CV splitter`, - An iterable yielding (train, test) splits as arrays of indices. For integer/None inputs, if ``y`` is binary or multiclass, :class:`~sklearn.model_selection.StratifiedKFold` is used. If the estimator is a classifier or if ``y`` is neither binary nor multiclass, :class:`~sklearn.model_selection.KFold` is used. Refer :ref:`User Guide ` for the various cross-validation strategies that can be used here. .. versionchanged:: 0.22 ``cv`` default value of None changed from 3-fold to 5-fold. :type cv: int, cross-validation generator or an iterable, default=None :param scoring: A string (see model evaluation documentation) or a scorer callable object / function with signature ``scorer(estimator, X, y)``. :type scoring: str, callable or None, default=None :param verbose: Controls verbosity of output. :type verbose: int, default=0 :param n_jobs: Number of cores to run in parallel while fitting across folds. ``None`` means 1 unless in a :obj:`joblib.parallel_backend` context. ``-1`` means using all processors. See :term:`Glossary ` for more details. .. versionadded:: 0.18 :type n_jobs: int or None, default=None :param importance_getter: If 'auto', uses the feature importance either through a `coef_` or `feature_importances_` attributes of estimator. Also accepts a string that specifies an attribute name/path for extracting feature importance. For example, give `regressor_.coef_` in case of :class:`~sklearn.compose.TransformedTargetRegressor` or `named_steps.clf.feature_importances_` in case of :class:`~sklearn.pipeline.Pipeline` with its last step named `clf`. If `callable`, overrides the default feature importance getter. The callable is passed with the fitted estimator and it should return importance for each feature. .. versionadded:: 0.24 :type importance_getter: str or callable, default='auto' .. attribute:: classes_ The classes labels. Only available when `estimator` is a classifier. :type: ndarray of shape (n_classes,) .. attribute:: estimator_ The fitted estimator used to select features. :type: ``Estimator`` instance .. attribute:: grid_scores_ The cross-validation scores such that ``grid_scores_[i]`` corresponds to the CV score of the i-th subset of features. .. deprecated:: 1.0 The `grid_scores_` attribute is deprecated in version 1.0 in favor of `cv_results_` and will be removed in version 1.2. :type: ndarray of shape (n_subsets_of_features,) .. attribute:: cv_results_ A dict with keys: split(k)_test_score : ndarray of shape (n_features,) The cross-validation scores across (k)th fold. mean_test_score : ndarray of shape (n_features,) Mean of scores over the folds. std_test_score : ndarray of shape (n_features,) Standard deviation of scores over the folds. .. versionadded:: 1.0 :type: dict of ndarrays .. attribute:: n_features_ The number of selected features with cross-validation. :type: int .. attribute:: n_features_in_ Number of features seen during :term:`fit`. Only defined if the underlying estimator exposes such an attribute when fit. .. versionadded:: 0.24 :type: int .. attribute:: feature_names_in_ Names of features seen during :term:`fit`. Defined only when `X` has feature names that are all strings. .. versionadded:: 1.0 :type: ndarray of shape (`n_features_in_`,) .. attribute:: ranking_ The feature ranking, such that `ranking_[i]` corresponds to the ranking position of the i-th feature. Selected (i.e., estimated best) features are assigned rank 1. :type: narray of shape (n_features,) .. attribute:: support_ The mask of selected features. :type: ndarray of shape (n_features,) .. seealso:: :obj:`RFE` Recursive feature elimination. .. rubric:: Notes The size of ``grid_scores_`` is equal to ``ceil((n_features - min_features_to_select) / step) + 1``, where step is the number of features removed at each iteration. Allows NaN/Inf in the input if the underlying estimator does as well. .. rubric:: References .. [1] Guyon, I., Weston, J., Barnhill, S., & Vapnik, V., "Gene selection for cancer classification using support vector machines", Mach. Learn., 46(1-3), 389--422, 2002. .. rubric:: Examples The following example shows how to retrieve the a-priori not known 5 informative features in the Friedman #1 dataset. >>> from sklearn.datasets import make_friedman1 >>> from sklearn.feature_selection import RFECV >>> from sklearn.svm import SVR >>> X, y = make_friedman1(n_samples=50, n_features=10, random_state=0) >>> estimator = SVR(kernel="linear") >>> selector = RFECV(estimator, step=1, cv=5) >>> selector = selector.fit(X, y) >>> selector.support_ array([ True, True, True, True, True, False, False, False, False, False]) >>> selector.ranking_ array([1, 1, 1, 1, 1, 6, 4, 3, 2, 5]) .. py:attribute:: estimator .. py:attribute:: step .. py:attribute:: importance_getter .. py:attribute:: cv .. py:attribute:: scoring .. py:attribute:: verbose .. py:attribute:: n_jobs .. py:attribute:: min_features_to_select .. py:method:: fit(X: numpy.ndarray, y: numpy.ndarray, train_idx_list: List[List[int]], val_idx_list: List[List[int]], groups: None = None, sample_weight_list: Optional[List[List[float]]] = None) -> RFECV Fit the RFE model and automatically tune the number of selected features. :param X: Training vector, where `n_samples` is the number of samples and `n_features` is the total number of features. :type X: {array-like, sparse matrix} of shape (n_samples, n_features) :param y: Target values (integers for classification, real numbers for regression). :type y: array-like of shape (n_samples,) :param groups: Group labels for the samples used while splitting the dataset into train/test set. Only used in conjunction with a "Group" :term:`cv` instance (e.g., :class:`~sklearn.model_selection.GroupKFold`). .. versionadded:: 0.20 :type groups: array-like of shape (n_samples,) or None, default=None :returns: **self** -- Fitted estimator. :rtype: object .. py:property:: grid_scores_