Title: | Multiple Class Area under ROC Curve |
---|---|
Description: | An S4 implementation of Eq. (3) and Eq. (7) by David J. Hand and Robert J. Till (2001) <DOI:10.1023/A:1010920819831>. |
Authors: | Andreas Dominik Cullmann [aut, cre] |
Maintainer: | Andreas Dominik Cullmann <[email protected]> |
License: | BSD_2_clause + file LICENSE |
Version: | 1.0.1.9000 |
Built: | 2025-03-11 03:45:39 UTC |
Source: | https://gitlab.com/fvafrcu/handtill2001 |
A very lean package implementing merely given by Hand and Till
(2001), Eq. (7).
given by Hand and Till (2001) defines a multiple class
version of the area under curve of the receiver operating characteristic.
David J. Hand and Robert J. Till (2001). A Simple Generalisation of the Area Under the ROC Curve for Multiple Class Classification Problems. Machine Learning 45(2), p. 171–186. DOI: 10.1023/A:1010920819831.
help(package="HandTill2001")
, especially
?HandTill2001::auc
; various packages that calculate
binary class AUC (ROCR
) or multiple class AUC
(pROC, caTools
).
library(HandTill2001) data(ht01.multipleclass) auc( multcap( response = ht01.multipleclass$observed, predicted = as.matrix(ht01.multipleclass[, levels(ht01.multipleclass$observed)]) ) )
library(HandTill2001) data(ht01.multipleclass) auc( multcap( response = ht01.multipleclass$observed, predicted = as.matrix(ht01.multipleclass[, levels(ht01.multipleclass$observed)]) ) )
auc
in Package HandTill2001
Calculate area under curve of the receiver operating characteristic for two or more prediction classes.
## S4 method for signature 'bincap' auc(object) ## S4 method for signature 'multcap' auc(object)
## S4 method for signature 'bincap' auc(object) ## S4 method for signature 'multcap' auc(object)
object |
An object of |
Depending on whether object
is of class bincap
or of class
multcap
, a binary class or multiple class AUC is calculated.
An object of class "numeric"
.
calculates the AUC statistic for a binary class response following Hand and Till (2001), Eq. (3).
calculates the AUC statistic for a multiple class response following Hand and Till (2001), Eq. (7).
David J. Hand and Robert J. Till (2001). A Simple Generalisation of the Area Under the ROC Curve for Multiple Class Classification Problems. Machine Learning 45(2), p. 171–186. DOI: 10.1023/A:1010920819831.
data(ht01.twoclass, package = "HandTill2001") message(" == AUC for a binary class response") message(" == == HandTill2001 result:") HandTill2001::auc(HandTill2001::bincap( response = as.factor(ht01.twoclass[["observed"]]), predicted = ht01.twoclass[["predicted"]], true = "1" )) ## Not run: message(" == == ROCR result:") ROCR::performance(ROCR::prediction( labels = ht01.twoclass[["observed"]], predictions = ht01.twoclass[["predicted"]] ), measure = "auc" )@y.values ## End(Not run) data(ht01.multipleclass, package = "HandTill2001") message(" == AUC for a multiple class response") predicted <- as.matrix(ht01.multipleclass[, levels(ht01.multipleclass[["observed"]])]) HandTill2001::auc(HandTill2001::multcap( response = ht01.multipleclass[["observed"]], predicted = predicted ))
data(ht01.twoclass, package = "HandTill2001") message(" == AUC for a binary class response") message(" == == HandTill2001 result:") HandTill2001::auc(HandTill2001::bincap( response = as.factor(ht01.twoclass[["observed"]]), predicted = ht01.twoclass[["predicted"]], true = "1" )) ## Not run: message(" == == ROCR result:") ROCR::performance(ROCR::prediction( labels = ht01.twoclass[["observed"]], predictions = ht01.twoclass[["predicted"]] ), measure = "auc" )@y.values ## End(Not run) data(ht01.multipleclass, package = "HandTill2001") message(" == AUC for a multiple class response") predicted <- as.matrix(ht01.multipleclass[, levels(ht01.multipleclass[["observed"]])]) HandTill2001::auc(HandTill2001::multcap( response = ht01.multipleclass[["observed"]], predicted = predicted ))
bincap
bincap(...)
is an alias to new("bincap", ...)
.
bincap(response, predicted, true = "1")
bincap(response, predicted, true = "1")
response |
Object of class |
predicted |
Object of class |
true |
Object of class |
There is no casting or conversion of data. bincap(...)
is just an
alias to new("bincap", ...)
.
An object of class bincap
.
library(HandTill2001) data(ht01.twoclass) str(ht01.twoclass$observed) message("note that ht01.twoclass$observed is not a factor; we have to convert it.") bincap( response = as.factor(ht01.twoclass$observed), predicted = ht01.twoclass$predicted, true = c("1") )
library(HandTill2001) data(ht01.twoclass) str(ht01.twoclass$observed) message("note that ht01.twoclass$observed is not a factor; we have to convert it.") bincap( response = as.factor(ht01.twoclass$observed), predicted = ht01.twoclass$predicted, true = c("1") )
S4 class for a binary class response and corresponding (predicted) probabilities.
Objects can be created by calls of the form
new("bincap", ...)
. They are used to store a binary class response
(one
of the two levels
of which is supposed to be true
), the
information which of the two levels
of the binary class response is
thought of as 'true'/'positive'/'present' (the other one would then be
thought of as 'false'/'negative'/'absence') and the predicted probabilities
that response
is true
.
No defaults are set. Especially, you have to explicitly initialize
true
, there is no trying to guess it from the levels
of
response
.
class?HandTill2001::cap
,
class?HandTill2001::multcap
,
?HandTill2001::bincap
showClass("bincap")
showClass("bincap")
Multiple class data and probability predictions thereof.
A data frame with 214 observations on the following 7 variables.
a factor with levels
Con
Head
Tabl
Veh
WinF
WinNF
a numeric vector
a numeric vector
a numeric vector
a numeric vector
a numeric vector
a numeric vector
Multiple class data ('observed': MASS::fgl$type
) and probability
predictions (predict(fgl.rp4)
, cf. Venables and Ripley (2002), p. 264
and ‘Source’) from rpart::rpart
.
## From: Forensic glass example Venables and Ripley (2002) pp. 261--265 library(MASS);library(rpart);data(fgl);set.seed(123) fgl.rp4 <- rpart(type ~ ., data = fgl, cp = 0.03 , parms = list(split = "information")) ht01.multipleclass <- data.frame(observed = fgl$type, predict(fgl.rp4)) write.table(ht01.multipleclass, file = "ht01.multipleclass.txt")
Venables, W. N and Ripley, B. D. (2002), Modern Applied Statistics with S (4th edition). Springer, ISBN 0-387-95457-0
library(HandTill2001) data(ht01.multipleclass) str(ht01.multipleclass)
library(HandTill2001) data(ht01.multipleclass) str(ht01.multipleclass)
Binary class data and probability predictions thereof.
A data frame with 189 observations on the following 2 variables.
a numeric vector
a numeric vector
Binary class data ('observed': MASS::birthwt$low
) and probability
predictions
(predict(birthwt.step2, type = "response")
, cf. Venables and Ripley
(2002), pp. 195f and ‘Source’) from stats::glm
.
## From: A binary class data example Venables and Ripley pp. 194--199 library(MASS); data("birthwt"); attach(birthwt) race <- (factor(race, labels = c("white", "black", "other"))) ptd <- factor(ptl > 0) ftv <- factor(ftv) levels(ftv)[-(1:2)] <- "2+" bwt <- data.frame(low = factor(low), age, lwt, race, smoke = (smoke > 0) , ptd, ht = (ht > 0), ui = (ui > 0), ftv) detach(birthwt) birthwt.glm <- glm(low ~ ., family=binomial(link=logit), data=bwt) birthwt.step2 <- stepAIC(birthwt.glm, ~ .^2 + I(scale(age)^2) + I(scale(lwt)^2), trace = F ) ht01.twoclass <- data.frame(observed = bwt$low , predicted = predict(birthwt.step2 , type = "response")) write.table(ht01.twoclass, file = "ht01.twoclass.txt")
Venables, W. N and Ripley, B. D. (2002), Modern Applied Statistics with S (4th edition). Springer, ISBN 0-387-95457-0
library(HandTill2001) data(ht01.twoclass) str(ht01.twoclass)
library(HandTill2001) data(ht01.twoclass) str(ht01.twoclass)
multcap
multcap(...)
is an alias to new("multcap", ...)
.
multcap(response, predicted)
multcap(response, predicted)
response |
Object of class |
predicted |
Object of class |
There is no casting or conversion of data. multcap(...)
is just
an alias to new("multcap", ...)
.
An object of class multcap
.
library(HandTill2001) data(ht01.multipleclass) str(ht01.multipleclass$observed) message("note that ht01.multipleclass$observed is a factor; we do not have to convert it.") multcap( response = ht01.multipleclass$observed, predicted = as.matrix(ht01.multipleclass[, levels(ht01.multipleclass$observed)]) )
library(HandTill2001) data(ht01.multipleclass) str(ht01.multipleclass$observed) message("note that ht01.multipleclass$observed is a factor; we do not have to convert it.") multcap( response = ht01.multipleclass$observed, predicted = as.matrix(ht01.multipleclass[, levels(ht01.multipleclass$observed)]) )
S4 class for a multiple class response and corresponding (predicted) probabilities.
Objects can be created by calls of the form
new("multcap", ...)
. They are used to store a multiple class response
and the predicted probabilities for each of the levels(response)
.
class?HandTill2001::cap
,
class?HandTill2001::bincap
,
?HandTill2001::multcap
showClass("multcap")
showClass("multcap")