Devel::Cover - Code coverage metrics for Perl
version 1.36
To get coverage for an uninstalled module:
cover -test
or
cover -delete
HARNESS_PERL_SWITCHES=-MDevel::Cover make test
cover
To get coverage for an uninstalled module which uses the Module::Build manpage (0.26 or
later):
./Build testcover
If the module does not use the t/*.t framework:
PERL5OPT=-MDevel::Cover make test
If you want to get coverage for a program:
perl -MDevel::Cover yourprog args
cover
To alter default values:
perl -MDevel::Cover=-db,cover_db,-coverage,statement,time yourprog args
This module provides code coverage metrics for Perl. Code coverage metrics
describe how thoroughly tests exercise code. By using Devel::Cover you can
discover areas of code not exercised by your tests and determine which tests
to create to increase coverage. Code coverage can be considered an indirect
measure of quality.
Although it is still being developed, Devel::Cover is now quite stable and
provides many of the features to be expected in a useful coverage tool.
Statement, branch, condition, subroutine, and pod coverage information is
reported. Statement and subroutine coverage data should be accurate. Branch
and condition coverage data should be mostly accurate too, although not always
what one might initially expect. Pod coverage comes from the Pod::Coverage manpage.
If the Pod::Coverage::CountParents manpage is available it will be used instead.
Coverage data for other criteria are not yet collected.
The cover program can be used to generate coverage reports. Devel::Cover
ships with a number of reports including various types of HTML output, textual
reports, a report to display missing coverage in the same format as compilation
errors and a report to display coverage information within the Vim editor.
It is possible to add annotations to reports, for example you can add a column
to an HTML report showing who last changed a line, as determined by git blame.
Some annotation modules are shipped with Devel::Cover and you can easily
create your own.
The gcov2perl program can be used to convert gcov files to Devel::Cover
databases. This allows you to display your C or XS code coverage together
with your Perl coverage, or to use any of the Devel::Cover reports to display
your C coverage data.
Code coverage data are collected by replacing perl ops with functions which
count how many times the ops are executed. These data are then mapped back to
reality using the B compiler modules. There is also a statement profiling
facility which should not be relied on. For proper profiling use
the Devel::NYTProf manpage. Previous versions of Devel::Cover collected coverage data by
replacing perl's runops function. It is still possible to switch to that mode
of operation, but this now gets little testing and will probably be removed
soon. You probably don't care about any of this.
The most appropriate mailing list on which to discuss this module would be
perl-qa. See http://lists.perl.org/list/perl-qa.html.
The Devel::Cover repository can be found at
http://github.com/pjcj/Devel--Cover. This is also where problems should be
reported.
- B::Debug
This was core before Perl 5.30.0.
By adding use Devel::Cover; to your mod_perl startup script, you should be
able to collect coverage information when running under mod_perl. You can
also add any options you need at this point. I would suggest adding this as
early as possible in your startup script in order to collect as much coverage
information as possible.
Alternatively, add -MDevel::Cover to the parameters for mod_perl.
In this example, Devel::Cover will be operating in silent mode.
PerlSwitches -MDevel::Cover=-silent,1
-blib - "use blib" and ignore files matching \bt/ (default true
if blib directory exists, false otherwise)
-coverage criterion - Turn on coverage for the specified criterion. Criteria
include statement, branch, condition, path, subroutine,
pod, time, all and none (default all available)
-db cover_db - Store results in coverage db (default ./cover_db)
-dir path - Directory in which coverage will be collected (default
cwd)
-ignore RE - Set regular expressions for files to ignore (default
"/Devel/Cover\b")
+ignore RE - Append to regular expressions of files to ignore
-inc path - Set prefixes of files to include (default @INC)
+inc path - Append to prefixes of files to include
-loose_perms val - Use loose permissions on all files and directories in
the coverage db so that code changing EUID can still
write coverage information (default off)
-merge val - Merge databases, for multiple test benches (default on)
-select RE - Set regular expressions of files to select (default none)
+select RE - Append to regular expressions of files to select
-silent val - Don't print informational messages (default off)
-subs_only val - Only cover code in subroutine bodies (default off)
-replace_ops val - Use op replacing rather than runops (default on)
-summary val - Print summary information if val is true (default on)
You can specify options to some coverage criteria. At the moment only pod
coverage takes any options. These are the parameters which are passed into
the the Pod::Coverage manpage constructor. The extra options are separated by dashes,
and you may specify as many as you wish. For example, to specify that all
subroutines containing xx are private, call Devel::Cover with the option
-coverage,pod-also_private-xx.
Or, to ignore all files in t/lib as well as files ending in Foo.pm:
cover -test -silent -ignore ^t/lib/,Foo.pm$
Note that -ignore replaces any default ignore regexes. To preserve any
ignore regexes which have already been set, use +ignore:
cover -test -silent +ignore ^t/lib/,Foo.pm$
You may select the files for which you want to collect coverage data using the
select, ignore and inc options. The system uses the following procedure to
decide whether a file will be included in coverage reports:
You may add to the REs to select by using +select, or you may reset the
selections using -select. The same principle applies to the REs to ignore.
The inc directories are initially populated with the contents of perl's @INC
array. You may reset these directories using -inc, or add to them using +inc.
Although these options take regular expressions, you should not enclose the RE
within // or any other quoting characters.
The options -coverage, [+-]select, [+-]ignore and [+-]inc can be specified
multiple times, but they can also take multiple comma separated arguments. In
any case you should not add a space after the comma, unless you want the
argument to start with that literal space.
Sometimes you have code which is uncoverable for some reason. Perhaps it is
an else clause that cannot be reached, or a check for an error condition that
should never happen. You can tell Devel::Cover that certain criteria are
uncoverable and then they are not counted as errors when they are not
exercised. In fact, they are counted as errors if they are exercised.
This feature should only be used as something of a last resort. Ideally you
would find some way of exercising all your code. But if you have analysed
your code and determined that you are not going to be able to exercise it, it
may be better to record that fact in some formal fashion and stop Devel::Cover
complaining about it, so that real problems are not lost in the noise.
If you have uncoverable criteria I suggest not using the default HTML report
(with uses html_minimal at the moment) because this sometimes shows uncoverable
points as uncovered. Instead, you should use the html_basic report for HTML
output which should behave correctly in this regard.
There are two ways to specify a construct as uncoverable, one invasive and one
non-invasive.
You can use special comments in your code to specify uncoverable criteria.
Comments are of the form:
# uncoverable <criterion> [details]
The keyword ``uncoverable'' must be the first text in the comment. It should be
followed by the name of the coverage criterion which is uncoverable. There
may then be further information depending on the nature of the uncoverable
construct.
The ``uncoverable'' comment should appear on either the same line as the
statement, or on the line before it:
$impossible++; # uncoverable statement
# uncoverable statement
it_has_all_gone_horribly_wrong();
If there are multiple statements (or any other criterion) on a line you can
specify which statement is uncoverable by using the ``count'' attribute,
count:n, which indicates that the uncoverable statement is the nth statement
on the line.
# uncoverable statement count:1
# uncoverable statement count:2
cannot_run_this(); or_this();
The ``uncoverable'' comment should specify whether the ``true'' or ``false'' branch
is uncoverable.
# uncoverable branch true
if (pi == 3)
Both branches may be uncoverable:
# uncoverable branch true
# uncoverable branch false
if (impossible_thing_happened_one_way()) {
handle_it_one_way(); # uncoverable statement
} else {
handle_it_another_way(); # uncoverable statement
}
If there is an elsif in the branch then it can be addressed as the second
branch on the line by using the ``count'' attribute. Further elsifs are the
third and fourth ``count'' value, and so on:
# uncoverable branch false count:2
if ($thing == 1) {
handle_thing_being_one();
} elsif ($thing == 2) {
handle_thing_being_tow();
} else {
die "thing can only be one or two, not $thing"; # uncoverable statement
}
Because of the way in which Perl short-circuits boolean operations, there are
three ways in which such conditionals can be uncoverable. In the case of
$x && $y for example, the left operator may never be true, the right operator
may never be true, and the whole operation may never be false. These
conditions may be modelled thus:
# uncoverable branch true
# uncoverable condition left
# uncoverable condition false
if ($x && !$y) {
$x++; # uncoverable statement
}
# uncoverable branch true
# uncoverable condition right
# uncoverable condition false
if (!$x && $y) {
}
Or conditionals are handled in a similar fashion (TODO - provide some
examples) but xor conditionals are not properly handled yet.
As for branches, the ``count'' value may be used for either conditions in elsif
conditionals, or for complex conditions.
A subroutine should be marked as uncoverable at the point where the first
statement is marked as uncoverable. Ideally all other criteria in the
subroutine would be marked as uncoverable automatically, but that isn't the
case at the moment.
sub z {
# uncoverable subroutine
$y++; # uncoverable statement
}
If you can't, or don't want to add coverage comments to your code, you can
specify the uncoverable information in a separate file. By default the files
PWD/.uncoverable and HOME/.uncoverable are checked. If you use the
-uncoverable_file parameter then the file you provide is checked as well as
those two files.
The interface to managing this file is the cover program, and the options
are:
-uncoverable_file
-add_uncoverable_point
-delete_uncoverable_point **UNIMPLEMENTED**
-clean_uncoverable_points **UNIMPLEMENTED**
The parameter for -add_uncoverable_point is a string composed of up to seven
space separated elements: ``$file $criterion $line $count $type $class $note''.
The contents of the uncoverable file is the same, with one point per line.
The -silent option is turned on when Devel::Cover is invoked via
$HARNESS_PERL_SWITCHES or $PERL5OPT. Devel::Cover tries to do the right thing
when $MOD_PERL is set. $DEVEL_COVER_OPTIONS is appended to any options passed
into Devel::Cover.
Note that when Devel::Cover is invoked via an environment variable, any modules
specified on the command line, such as via the -Mmodule option, will not be
covered. This is because the environment variables are processed after the
command line and any code to be covered must appear after Devel::Cover has been
loaded. To work around this, Devel::Cover can also be specified on the command
line.
When running Devel::Cover's own test suite, $DEVEL_COVER_DEBUG turns on
debugging information, $DEVEL_COVER_GOLDEN_VERSION overrides Devel::Cover's
own idea of which golden results it should test against, and
$DEVEL_COVER_NO_COVERAGE runs the tests without collecting coverage.
$DEVEL_COVER_DB_FORMAT may be set to ``Sereal'', ``JSON'' or ``Storable'' to
override the default choice of DB format (Sereal, then JSON if either are
available, otherwise Storable). $DEVEL_COVER_IO_OPTIONS provides fine-grained
control over the DB format. For example, setting it to ``pretty'' when the
format is JSON will store the DB in a readable JSON format. $DEVEL_COVER_CPUS
overrides the automated detection of the number of CPUs to use in parallel
testing.
Some code and ideas cribbed from:
There are things that Devel::Cover can't cover.
Perl keeps track of which modules have been loaded (to avoid reloading
them). Because of this, it isn't possible to get coverage for a path
where a runtime import fails if the module being imported is one that
Devel::Cover uses internally. For example, suppose your program has
this function:
sub foo {
eval { require Storable };
if ($@) {
carp "Can't find Storable";
return;
}
# ...
}
You might write a test for the failure mode as
BEGIN { @INC = () }
foo();
# check for error message
Because Devel::Cover uses Storable internally, the import will succeed
(and the test will fail) under a coverage run.
Modules used by Devel::Cover while gathering coverage:
If you redefine a subroutine you may find that the original subroutine is not
reported on. This is because I haven't yet found a way to locate the original
CV. Hints, tips or patches to resolve this will be gladly accepted.
The module Test::TestCoverage uses this technique and so should not be used in
conjunction with Devel::Cover.
Almost certainly.
See the BUGS file, the TODO file and the bug trackers at
https://github.com/pjcj/Devel--Cover/issues?sort=created&direction=desc&state=open
and https://rt.cpan.org/Public/Dist/Display.html?Name=Devel-Cover
Please report new bugs on Github.
Copyright 2001-2019, Paul Johnson (paul@pjcj.net)
This software is free. It is licensed under the same terms as Perl itself.
The latest version of this software should be available on CPAN and from my
homepage: http://www.pjcj.net/.
|