How to programmatically skip a test in mocha?
mocha.jsmocha.js Problem Overview
I have a code where certain tests will always fail in CI environment. I would like to disable them based on an environment condition.
How to programmatically skip a test in mocha during the runtime execution?
mocha.js Solutions
Solution 1 - mocha.js
You can skip tests by placing an x in front of the describe or it block, or placing a .skip after it.
xit('should work', function (done) {});
describe.skip('features', function() {});
You can also run a single test by placing a .only on the test. for instance
describe('feature 1', function() {});
describe.only('feature 2', function() {});
describe('feature 3', function() {});
Only the feature 2 block would run in this case.
There doesn't appear to be a way to programmatically skip tests, but you could just do some sort of check in a beforeEach statement and only run the test if the flag was set.
beforeEach(function(){
if (wrongEnvironment){
runTest = false
}
}
describe('feature', function(){
if(runTest){
it('should work', function(){
// Test would not run or show up if runTest was false,
}
}
}
Solution 2 - mocha.js
Use Mocha's skip() function
It can be used to either statically to disable a test or entire suite, or dynamically skip it at runtime.
Here's an example runtime usage:
it('should only test in the correct environment', function() {
if (/* check test environment */) {
// make assertions
} else {
this.skip();
}
});
Solution 3 - mocha.js
This answer does work for ES6.
Instead of:
describe('your describe block', () => {
You want:
(condition ? describe : describe.skip)('your describe block', () => {
This conditionally skips all tests in the describe block IF the condition is false.
Or, instead of:
it('your it block', () => {
You want:
(condition ? it : it.skip)('your it block', () => {
This conditionally skips one test IF the condition is false.
Solution 4 - mocha.js
I use runtime skipping from Mocha for the same scenario as you're describing. It is the copy paste from the docs:
it('should only test in the correct environment', function() {
if (/* check test environment */) return this.skip();
// make assertions
});
As you can see, it skips the test based on environment. My own condition is if(process.env.NODE_ENV === 'continuous-integration').
Solution 5 - mocha.js
to skip tests, use describe.skip or it.skip
describe('Array', function() {
it.skip('#indexOf', function() {
// ...
});
});
to include tests you could use describe.only or it.only
describe('Array', function() {
it.only('#indexOf', function() {
// ...
});
});
More info at https://mochajs.org/#inclusive-tests
Solution 6 - mocha.js
It depends how you want to programmatically skip the test. If the conditions for skipping can be determined before any test code is run, then you can just call it or it.skip as needed, based on a condition. For instance, this will skip some tests if the environment variable ONE is set to any value:
var conditions = {
"condition one": process.env["ONE"] !== undefined
// There could be more conditions in this table...
};
describe("conditions that can be determined ahead of time", function () {
function skip_if(condition, name, callback) {
var fn = conditions[condition] ? it.skip: it;
fn(name, callback);
};
skip_if("condition one", "test one", function () {
throw new Error("skipped!");
});
// async.
skip_if("condition one", "test one (async)", function (done) {
throw new Error("skipped!");
});
skip_if("condition two", "test two", function () {
console.log("test two!");
});
});
If the conditions you want to check can only be determined at test time, it is a bit more complicated. If you do not want to access anything not strictly speaking part of the testing API, then you could do this:
describe("conditions that can be determined at test time", function () {
var conditions = {};
function skip_if(condition, name, callback) {
if (callback.length) {
it(name, function (done) {
if (conditions[condition])
done();
else
callback(done);
});
}
else {
it(name, function () {
if (conditions[condition])
return;
callback();
});
}
};
before(function () {
conditions["condition one"] = true;
});
skip_if("condition one", "test one", function () {
throw new Error("skipped!");
});
// async.
skip_if("condition one", "test one (async)", function (done) {
throw new Error("skipped!");
});
skip_if("condition two", "test two", function () {
console.log("test two!");
});
});
Whereas my first example was marking the tests as formally skipped (aka "pending"), the method I've just shown will just avoid performing the actual test but the tests won't be marked as formally skipped. They will be marked as passed. If you absolutely want to have them skipped I don't know of any way short of accessing parts that are not properly speaking part of the testing API:
describe("conditions that can be determined at test time", function () {
var condition_to_test = {}; // A map from condition names to tests.
function skip_if(condition, name, callback) {
var test = it(name, callback);
if (!condition_to_test[condition])
condition_to_test[condition] = [];
condition_to_test[condition].push(test);
};
before(function () {
condition_to_test["condition one"].forEach(function (test) {
test.pending = true; // Skip the test by marking it pending!
});
});
skip_if("condition one", "test one", function () {
throw new Error("skipped!");
});
// async.
skip_if("condition one", "test one (async)", function (done) {
throw new Error("skipped!");
});
skip_if("condition two", "test two", function () {
console.log("test two!");
});
});
Solution 7 - mocha.js
I am not sure if this qualifies as „programmatic skipping“, but in order to selectively skip some specific tests for our CI environment, I use Mocha's tagging feature (https://github.com/mochajs/mocha/wiki/Tagging).
In describe() or it() messages, you can add a tag like @no-ci. To exclude those tests, you could define a specific "ci target" in your package.json and use --grep and --invert parameters like:
"scripts": {
"test": "mocha",
"test-ci" : "mocha --reporter mocha-junit-reporter --grep @no-ci --invert"
}
Solution 8 - mocha.js
We can write a nice clean wrapper function to conditionally run tests as follows:
function ifConditionIt(title, test) {
// Define your condition here
return condition ? it(title, test) : it.skip(title, test);
}
This can then be required and used in your tests as follows:
ifConditionIt('Should be an awesome test', (done) => {
// Test things
done();
});
Solution 9 - mocha.js
You can use my package mocha-assume to skip tests programmatically, but only from outside the tests. You use it like this:
assuming(myAssumption).it("does someting nice", () => {});
Mocha-assume will only run your test when myAssumption is true, otherwise it will skip it (using it.skip) with a nice message.
Here's a more detailed example:
describe("My Unit", () => {
/* ...Tests that verify someAssuption is always true... */
describe("when [someAssumption] holds...", () => {
let someAssumption;
beforeAll(() => {
someAssumption = /* ...calculate assumption... */
});
assuming(someAssumption).it("Does something cool", () => {
/* ...test something cool... */
});
});
});
Using it this way, you can avoid cascading failures. Say the test "Does something cool" would always fail when someAssumption does not hold - But this assumption was already tested above (in Tests that verify someAssuption is always true").
So the test failure does not give you any new information. In fact, it is even a false-positive: The test did not fail because "something cool" did not work, but because a precondition for the test was not satisfied. with mocha-assume you can often avoid such false positives.
Solution 10 - mocha.js
This is not really using mocha's features, rather tweaking it to get the behaviour I wanted.
I wanted to skip any subsequent 'it's' in my protractor mocha tests and one 'it' failed. This was because once one step of a journey test failed it was almost certain the rest would fail, and may take a long time and hog the build server if they are using browser waits for elements to appear on a page etc.
When just running standard mocha tests (not protractor) this can be achieved with global beforeEach and afterEach hooks by attaching a 'skipSubsequent' flag to the test's parent (describe) like this:
beforeEach(function() {
if(this.currentTest.parent.skipSubsequent) {
this.skip();
}
});
afterEach(function() {
if (this.currentTest.state === 'failed') {
this.currentTest.parent.skipSubsequent = 'true'
}
})
When attempting this with protractor and mocha it the scope of 'this' has changed and the code above does not work. You end up with an error message like 'error calling done()' and protractor halts.
Instead I ended up with the code below. Not the prettiest, but it ends up replacing the implementation of remaining test functions with a this.skip(). This will probably stop working if/when the internals of mocha change with later versions.
It was figured out through some trial and error by debugging and inspecting mocha's internals...helps make browser test suites complete sooner when the tests fail though.
beforeEach(function() {
var parentSpec = this.currentTest.parent;
if (!parentSpec.testcount) {
parentSpec.testCount = parentSpec.tests.length;
parentSpec.currentTestIndex = 0;
} else {
parentSpec.currentTestIndex = parentSpec.currentTestIndex + 1;
}
if (parentSpec.skipSubsequent) {
parentSpec.skipSubsequent = false;
var length = parentSpec.tests.length;
var currentIndex = parentSpec.currentTestIndex;
for (var i = currentIndex + 1; i < length; i++) {
parentSpec.tests[i].fn = function() {
this.skip();
};
}
}
});
afterEach(function() {
if (this.currentTest.state === 'failed') {
this.currentTest.parent.skipSubsequent = 'true'
}
});
Solution 11 - mocha.js
can be used based on condition, for instance, declare a var and when the condition fails, use this.skip();
> Note that skip() does not work in arrow function
let shouldRun: boolean;
before(function(){
if ($('#nsErrorIframe').isDisplayed()) {
driver.switchToFrame($('#nsErrorIframe'));
if ($('.ns-error-wrapper').isDisplayed()) {
console.log('PAGE IS NOT AVAILABLE');
shouldRun = false;
if ( shouldRun === false) {
this.skip();
}
}
}
});
Solution 12 - mocha.js
We have some fluky tests in our test environment and sometimes to switch off ones, using this approach:
mocha --config ./config/parallelrc.cjs --parallel --jobs 3 -- tests/spec/**/index.js -g @flaky -i
We mark the flaky tests @flaky in tests describe and set special -g rule, it means that mocha runs only tests with @flaky tag, and the next use -i - it means inversion, so mocha run only tests NOT @flaky.
So, thinking it'll be useful for you)
Solution 13 - mocha.js
Say I wanted to skip my parametrized test if my test description contained the string "foo", I would do this:
// Skip parametrized test if description contains the string "foo"
(test.description.indexOf("foo") === -1 ? it : it.skip)("should test something", function (done) {
// Code here
});
// Parametrized tests
describe("testFoo", function () {
test({
description: "foo" // This will skip
});
test({
description: "bar" // This will be tested
});
});
In your case, I believe that if you wanted to check environment variables, you could use NodeJS's:
process.env.ENV_VARIABLE
For example (Warning: I haven't tested this bit of code!), maybe something like this:
(process.env.NODE_ENV.indexOf("prod") === -1 ? it : it.skip)("should...", function(done) {
// Code here
});
Where you can set ENV_VARIABLE to be whatever you are keying off of, and using that value, skip or run the test. (FYI the documentation for the NodeJS' process.env is here: https://nodejs.org/api/process.html#process_process_env)
I won't take complete credit for the first part of this solution, I found and tested the answer and it worked perfectly to skip tests based on a simple condition through this resource: https://github.com/mochajs/mocha/issues/591
Hope this helps! :)
Solution 14 - mocha.js
mocha test/ --grep <pattern>
Solution 15 - mocha.js
As @danielstjules answered here there is a way to skip test. @author of this topic has copied answer from github.com mochajs discussion, but there is no info in what version of mocha it available.
I'm using grunt-mocha-test module for integrating mocha test functionality in my project. Jumping to last (for now) version - 0.12.7 bring me mocha version 2.4.5 with implementation of this.skip().
So, in my package.json
"devDependencies": {
"grunt-mocha-test": "^0.12.7",
...
And then
npm install
And it make me happy with this hook:
describe('Feature', function() {
before(function () {
if (!Config.isFeaturePresent) {
console.log('Feature not configured for that env, skipping...');
this.skip();
}
});
...
it('should return correct response on AB', function (done) {
if (!Config.isABPresent) {
return this.skip();
}
...
Solution 16 - mocha.js
Please don't. A test that doesn't work consistently across environments should be acknowledged as such by your build infrastructure. And it can be very disorienting when the CI builds have a different number of tests run than local.
Also it screws up repeatability. If different tests run on the server and local I can have tests failing in dev and passing in CI or vice versa. There's no forcing function and I have no way to quickly and accurately correct a failed build.
If you must turn off tests between environments, instead of conditionally running tests, tag your tests and use a filter to eliminate tests that don't work in certain build targets. That way everybody knows what's going on and it tempers their expectations. It also lets everybody know that there's inconsistency in the test framework, and someone might have a solution that gets them running properly again. If you just mute the test they might not even know there's a problem.