Negation glob causes issues with the newest version (3.0.1)

[asgoth]

Negation glob causes issues with the newest version (3.0.1). The following still worked in version 2.3.11:

var micromatch = require('micromatch')
var fixtures = ["bar/bar"];
var result = micromatch(fixtures, ["foo/**", "!foo/baz"]);

I would expect result to be an empty array, but it is ["bar/bar"]. When I remove "!foo/baz" it works, so it seems to be connected.

Note: the same occurs if fixtures is a string (e.g. "bar/bar")

[jonschlinkert]

Thanks I'll take a look

[jonschlinkert]

k, I just wanted to do some double-checking. before I comment with my opinion I'd like to hear yours about this...

what should the result be for the following?

var micromatch = require('micromatch')
var fixtures = ["bar/bar"];
var result = micromatch(fixtures, ["!foo/baz"]);

@doowb or (@phated, @es128 or anyone else have thoughts on this?)

edit: comparisons, for discussion:

var micromatch = require('micromatch');
var multimatch = require('multimatch');
var minimatch = require('minimatch');

console.log(minimatch.match(['foo', 'bar'], '!baz'));
//=> [ 'foo', 'bar' ]
console.log(micromatch(['foo', 'bar'], '!baz'));
//=> [ 'foo', 'bar' ]
console.log(multimatch(['foo', 'bar'], '!baz'));
//=> []

edit 2: one more example, IMHO this one makes expected behavior more clear:

console.log(minimatch.match(['foo', 'bar'], '!bar'));
//=> [ 'foo' ]
console.log(micromatch(['foo', 'bar'], '!bar'));
//=> [ 'foo' ]
console.log(multimatch(['foo', 'bar'], '!bar'));
//=> []

edit 3: (only micromatch and multimatch, since minimatch doesn't take an array of patterns)

console.log(micromatch(['foo/bar', 'foo/baz'], ['!*/bar', 'foo/*'])); //=> [ 'foo/baz' ]
console.log(multimatch(['foo/bar', 'foo/baz'], ['!*/bar', 'foo/*'])); //=> [ 'foo/bar', 'foo/baz' ]

console.log(micromatch(['foo/bar', 'foo/baz'], ['foo/*', '!*/bar'])); //=> [ 'foo/baz' ]
console.log(multimatch(['foo/bar', 'foo/baz'], ['foo/*', '!*/bar'])); //=> [ 'foo/baz' ]

[doowb]

what should the result be for the following?

["bar/baz"] since there's only a negative glob pattern passed in and that pattern "does not" match any fixtures.

[jonschlinkert]

Actually, I just doubled checked and the behavior seems to be consistent with the previous version of micromatch (v2.3.11). e.g. nothing has changed.

So, basically, if you want to negate a string, you should put a negation pattern that matches what you want to negate.

var micromatch = require('micromatch');
var fixtures = ['bar/bar'];
var result = micromatch(fixtures, ['foo/**', '!bar/bar']);
console.log(result);
//=> []

I'm going to go ahead and close b/c I think this is expected behavior that is consistent with minimatch, but anyone from the org can reopen so we can discuss if there is disagreement on this.

[asgoth]

Thanks for the explanation. I'll take a look

[asgoth]

Still, it feels "weird", especially since it worked in a previous version (which might be by accident). I would expect it to be excluded since it is not part of 'foo/**'. Which is the behaviour when there is no negation in the glob array.

var micromatch = require('micromatch');
var fixtures = ['bar/bar'];
var result = micromatch(fixtures, ['foo/**']);
console.log(result);
//=> []

I have a library which spawns configuration for requirejs optimizer. I'm looping over a set of file paths to verify if they need to be included or not:

const shouldInclude = micromatch(moduleFile.relative, module.validates).length > 0;

Now I need to change it into something that checks it in two steps. First without negations and if that still matches, include the negations again:

let validatesWithoutNegations = removeNegations(module.validates);
const shouldInclude = micromatch(moduleFile.relative, validatesWithoutNegations).length > 0;
if (shouldInclude) {
  shouldInclude = micromatch(moduleFile.relative, module.validates).length > 0;
}

[jonschlinkert]

especially since it worked in a previous version

hmm, it would help if you could show which version it worked in. (edit: as I mentioned in an earlier comment, I tested your patterns against earlier versions of micromatch and the results were the same as they are now)

Now I need to change it into something that checks it in two steps.

Why? can you be more specific with actual globs you're using? (e.g. not simplistic contrived examples). There are many unit tests covering negation behavior if you want to look over those. I'd also be happy to help you optimize what you're doing, I just need something more detailed to help me understand specifically what you're doing.

[asgoth]

It worked in version 2.3.11.

I'll try to explain a bit better. I've created a library that bundles files (using r.js. It is a library that we use internally for several huge repositories. Here is a simple (unit test) example:

• subdir
  • file1.js
  • file2.js
• main.js
• other.js

These files are amd modules (requirejs), so with references to other files, e.g.

main.js

define(['other', 'subdir/file2'], function () { ... });

subdir/file2.js

define(['subdir/file1'], function () { ... });

So I bundle these files by passing a module configuration:

var modules = [
  {
    name: 'main',
    // this should result in an exclude of 'subdir/file1.js' (which is in 'subdir/file2')
    validates: ['subdir/**', '!subdir/file2.js', 'main.js']
  }
];

This should create a module file main.js which includes main.js itself and subdir2 (except file2).

I'm doing this by reading the module file(s) (in this case main.js) and detecting (using amdetective) which files it uses (recursively). The files which are detected are validated against the validates glob above.

Now the issue which arises is that when validating other.js against the validates glob, it matches against the glob (it doesn't with version 2.3.11) and thus is included in the main.js bundle, which shouldn't.

Without the negation glob ('!subdir/file2.js'), other.js is not matched against the validates glob (but then of course subdir/file2.js matches).

[jonschlinkert]

It worked in version 2.3.11.

It didn't though. I just tested it again.

I'm guessing the paths are just relative of basenames at that point? Otherwise you'll need to do:

 validates: ['subdir/**', '!**/subdir/file2.js', 'main.js']

[asgoth]

Yeah, all paths are relative.

Behaviour is now different than multimatch. Just did a quick test:

const micromatch = require('micromatch'); // 3.0.1
const multimatch = require('multimatch');

console.log("micromatch: " + micromatch("other.js", ["subdir/**", "!subdir/file1.js", "main.js"]));
//=> ["other.js"]

console.log("multimatch: " + multimatch("other.js", ["subdir/**", "!subdir/file1.js", "main.js"]));
//=> []

const micromatch = require('micromatch'); // 2.3.11
const multimatch = require('multimatch');

console.log("micromatch: " + micromatch("other.js", ["subdir/**", "!subdir/file1.js", "main.js"]));
//=> []

console.log("multimatch: " + multimatch("other.js", ["subdir/**", "!subdir/file1.js", "main.js"]));
//=> []

So it looks weird to me that you say it behaved the same in version 2.3.11. Btw, my Node version is 6.9.5 (dunno if that matters).

[jonschlinkert]

Behaviour is now different than multimatch. Just did a quick test:

behavior should have always been different than multimatch, because (IMHO) multimatch has always been wrong. If micromatch was producing the same value as multimatch before, I believe that was a bug that has now been fixed.

that said... hmm I'm wondering if there is indeed a bug with a single file when multiple patterns are passed and a negation pattern is given. I'm not sure why you keep using a single file in the examples, but maybe that's what the bug is - if there is one. I'll look into it.

[jonschlinkert]

actually, I don't think it's single files, I think it's something else... something about this seems familiar... I'm looking into it now.

edit: yeah I think I know what it is. It's not a "glob matching" but, I think it's a bug in one of the loops.

okay, sorry for the comment bombs... forget everything I just said. There is no bug, however... I understand if it looks like one if you expected a different result. I know what's causing it and I have a solution for it. it's this code, which IMHO is the right way to go about it... at least more often than not. However, this is globbing and I've learned the hard way that what seems totally correct in one scenario will not seem correct in another scenario. For example, mm('foo', '!bar') sure seems like ['foo'] should be returned. But if you add more patterns to the array it gets murky.

As for the solution, for the sanity of micromatch users I'm going to remove that code (and believe it or not it only breaks two unit tests out of 36,000, so the code wasn't being used enough to justify it anyway). I'll push the fix up in a few min.

PS. I did go back and test a previous version of micromatch again and you are correct, it was returning a different result. Honest, I did it earlier and received the same result with different patterns, but it's possible I was still running a cached version or something. no idea.

[jonschlinkert]

Ok, let me know if that works for you

[asgoth]

Alright, this indeed works. Thank you for your quick fixing :)

[jonschlinkert]

no prob. thanks for your patience (with me) lol. I'm slow sometimes... I'm glad it's working for you now!

[asgoth]

Understandable :D Working for months on a new (major) version and then someone complains it doesn't work the minute it is finished :)

Anyways thanks

[jonschlinkert]

ha lol, actually I appreciate the issues. this was just a specific thing that I really thought I had a handle on. but it's a good reminder not to make assumptions. the more I learn the more I realize how much I really don't know lol