Skip to content

Commit

Permalink
Synchronizing from upstream master (#2)
Browse files Browse the repository at this point in the history
* Fix Slack, HipChat, and a few android browsers.

Fix Slack Desktop Client detection on Windows and Mac.
Add support for HipChat on Mac and Windows.
Fix Macthon and Crosswalk support on Android.

* add Qwantify spider

* Modified Catchpoint regex to account for Catchpoint or Catchpoint bot

* Resolve conflicts

* [ua-parser#136] Fixing detection error for Lumia

* [ua-parser#233] fix for Edge 12

* Add support for Chrome Mobile WebView detection

* Missed a property during merge, added back now

* Add several popular RSS bots to regexes.yaml

* Added new relic pinger bot

* fixed some case on gionee device

* Fix version parsing for Facebook and Pinterest app

The patch number was wrongly captured.

* Added Pinterestbot to differentiate between the app and the bot

* Podcast catcher applications

* Add support for Box Drive, Sync, and Notes

* - removing node 0.12 as mocha discontinued support
- adding node v6.x and v8.x

* Parse Headless Chrome versions

The initial implementation of the Headless Chrome detection (ua-parser#228) didn't detect
versions of the headless version. This has been fixed.

A test for Ubuntu Chromium in headless mode was also added. I haven't made it
recognized as HeadlessChromium as it has HeadlessChrome in the user agent. It
might be worthwhile to check how a pure compiled from source Chromium behaves.
Nevertheless, this commit is still an improvement.

Ref ua-parser#228
Ref karma-runner/karma#2762

* Add a test for a bare Chrome 60 instance

* Add Evolution CardDav/CalDav user agent

* Add Roundcube CardDav plugin user agent

* Add DAVdroid

* Add DAVdroid F-Droid variant

* Add Nextcloud desktop sync client

* Add Nextcloud android app

* add test for applenews browser

* Add regex to detect AppleNews as WKWebView

* Fix test and add new case for iPhone

* Fix regexps and tests for recognizing windows majors better

* Point to reference implementation where there is replacements supported with minor

* Update package.json to point public repo

* Update patterns for IE compatibility mode

* iOS 10.3.x and 11.x recognition

* Update package.json

* Update package.json

* Add default regex to detect iOS for unknown versions

* Fixing yaml syntax in podcasting_user_agent_strings.yaml

Symfony's yaml parser choked on these three lines since they're missing the colon.

* Update regexes.yaml to include Tableau

* add test case for Tableau

* Update regexes.yaml

Add support for CFNetwork macOS Apps on macOS v. 10.13

* Update test_os.yaml

Added test

* Add support for OktaMobile

* add some more matching for RCA phones

* little fix

* Fix merge conflict

* Added support for Outlook >= 2.62.0 running on iOS

* Should account for Trident/8.0 - "Compatibility view of Win10 with Internet Explorer 11"

* Fix ESPN Radio podcast catcher regex

Make it more precise and compatible with Boost.Regex syntax as well.

* Add "Instagram" app

* Add "Flipboard" (iOS,Android) and "Onefootball" (Android) apps

* updated test cases for SznProhlizec; new family name

* Replace empty alternations by optional group

A popular regexp implementation of Rust does not allow empty
alternations.

* Add "Flipboard-Briefing" and tests

* Add ESPN Radio podcast tests to test_ua.yaml

* Add Waterfox/Basilisk Firefox variants

* Removed .project from .gitignore

* fix invalid indentation in yaml

* Add HTTP clients for Go, Scala, Rust (reqwest)

* Add clients to access AWS S3 (Simple Storage Service)

* Add cloud storage clients: https://cyberduck.io/,
https://rclone.org/ and http://s3browser.com/

* Add download clients lftp (https://lftp.yar.ru/),
Axel (https://github.com/axel-download-accelerator/axel),
aria2 (https://aria2.github.io/) and PycURL (http://pycurl.io/)

* Add variants of Linux OS sent via Wget

* Add Naver Yeti and Daum to Spiders regex

* Added OS and UserAgent parser for Salesforce UA on Android

* Add support for patch_minor when detecting Chrome.

* Replace empty alternations by optional group

A popular regexp implementation of Rust does not allow empty
alternations.

* Add Naver Yeti and Daum to Spiders regex

* add support for mac os detection when agent is macoutlook

* Add patch_minor to other Chrome test cases

* Update Android Regex to allow for single digit OS versions

The current Android 9 user agent is as follows
Mozilla/5.0 (Linux; Android 9; Pixel Build/PPP3.180510.008) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.81 Mobile Safari/537.36
This commit allows the code to parse 9 as a valid OS version.

* Added Apple Watch and HomePod devices

* Added tests for Apple Watch & HomePod

* Fixed test

* Fix ua-parser#326: add insomnia user-agent

* iOS CFNetwork recognition update

- 9.3.x
- 10.2.1
- 11.2.6
- 11.3
- 11.4.x
- 12

Signed-off-by: Vitaliy Khamin <vitaliykhamin@gmail.com>

* Support WebView detection in Android KitKat to Lollipop

* Fix to identify MI devices

Dalvik/2.1.0 (Linux; U; Android 8.0.0; Mi A1 MIUI/V9.5.13.0.ODHMIFA)
Dalvik/2.1.0 (Linux; U; Android 5.0.2; Mi 4i MIUI/V8.1.7.0.LXIMIDI)

* Added Yahoo Mail Proxy to regexes.yaml

* Add test.

* Fix ua string in test

* Fix actual UA. Happy Monday

* Add support for Whale browser

* Fixed Google Pixel detection

Added tests

* Fix indenting in regexes.yaml

The additional spaces may cause IDEs (specifically IntelliJ in my case) to transform this into a tab character. As a result of this, Java can not properly parse the YAML file, and every look up throws the following exception `org.yaml.snakeyaml.scanner.ScannerException`.

By placing the comment on the expected level, parsing works as intended

* Add parser for Citrix Chrome app on Chrome OS

* Fix tests formatting and regex formatting

* Fix ua-parser#271: add Google user-agent on iOS

* Handle useragents that present Windows 10

* Added Snapchat detection

* Detect Facebook without FBAV version

Added test

* Fixed typo

* Added support for Facebook Messenger

* update salesforce regex

handle single digit android versions

* update salesforce tests

test single digit android versions

* split windows 8.1 version

* Add @commenthol to maintainers

* Update regexes.yaml

Lots of missing escapes on `.`s; I only did a few of them.

* safe regexes

* Ask users to test for ReDoS in CONTRIBUTING.md

* Safeify FB_IAB

* fix os for salesforce agent

* add regexes tests to npm test

* uap-ref-impl needs a new version published on npm

* remove unnecessary group

* remove node v4+v6, adding v10

incompatible module dependencies

* npmignore

* 0.6.0

* Adding tests and rules for PTST Webpagetest.org spiders

* Add LINE Browser

* User agent detection: ViaFree app

* OS detection: tvOS in apps

* Test cases

* Fix ReDoS vulnerability

* 0.6.1

* Fix tvOS regex

* 0.6.2

* fix: prevent backtracking in crawler, bot regexes

* 0.6.3

* Formatting regex for proper extraction and tests

* Setting expected value for device model properly

* Parse iPhone, iPad, and iPod as iOS devices

There is already a parser that uses this same regex for detecting
Mobile Safari Webviews, which we should also identify as iOS devices.

* Fixed regexes matching some user agent on windows in order avoid embedding the version number into the family string

* 0.6.4

* Add UA for GSA ("Google Search App") on Android, plus tests

* Fixes as per
ua-parser#372 (comment)

Also removed the example UA strings as they are in the tests

* Added Skype for Business detection

* Adding Datanyze UA and tests

* Removing device_replacement from UA parser

* Tidying up UA parser and adding test for OS

* Fixing user_agent_parsers replacements

* add MiuiBrowser and Mint Browser regexes

* 0.6.5

* Add Microsoft Office

* Add Ghost User-Agent String and tests

* Handling of cases where PTST doesn't have minor version

* device regexp simplification

* Add Sailfish OS and Sailfish Browser (ua-parser#391)

* Fixes case where Palo Alto identifies with multiple versions of Windows.

* 0.6.6

* 0.6.7

* Added backers and sponsors on the README

* Add Sumsung SC-01F

* New browser versions remove OS build info from user agent
https://www.chromestatus.com/feature/4558585463832576

* added Edge with Chromium UA and test

* fix test

* support Apple Watch devices

* Add regex for Xiaomi MIX, MIX 2 and MIX 2S.

http://www.fynas.com/ua/search?d=&b=MIUI%E6%B5%8F%E8%A7%88%E5%99%A8&k=MIX&page=1

* adding patch and minor patch to edge. instead of adding a regex for edge with crome modifued the existing one

* fixing tests

* formatting

* Add regex for chrome browser in Samsung Android phones.

native browser in Samsung device is always ok.
Mozilla/5.0 (Linux; Android 8.0.0; SAMSUNG SM-G9500 Build/R16NW) AppleWebKit/537.36 (KHTML, like Gecko) SamsungBrowser/9.3 Chrome/67.0.3396.87 Mobile Safari/537.36

But chrome browser was parsed to Generic SmartPhone, here's the UA:
Mozilla/5.0 (Linux; Android 8.0.0; SM-G9500) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.110 Mobile Safari/537.36

* Moved Facebook rules before "Bot general matcher"

We noticed UAs from Orange that contained OrangeBotswana that were caught
by the bot filter.

* remove prepended 'User agent:' from User-Agent in test_ua.yaml

* 0.6.8
  • Loading branch information
russellwhitaker committed Jun 19, 2019
1 parent 569a93f commit bfdad6a
Show file tree
Hide file tree
Showing 18 changed files with 6,191 additions and 560 deletions.
5 changes: 4 additions & 1 deletion .gitignore
@@ -1,2 +1,5 @@
.DS_Store
node_modules/
node_modules/
package-lock.json
*.tgz
*.log
8 changes: 8 additions & 0 deletions .npmignore
@@ -0,0 +1,8 @@
node_modules/
test_resources/
tests/
tmp/
*.bak
*.tgz
*.log
.*
1 change: 1 addition & 0 deletions .npmrc
@@ -0,0 +1 @@
package-lock=false
7 changes: 4 additions & 3 deletions .travis.yml
@@ -1,12 +1,13 @@
sudo: false
language: node_js
node_js:
- 0.12
- 4.0
- 8
- 10
- node

script:
- "npm test"

notifications:
irc: "chat.freenode.net#ua-parser"

7 changes: 4 additions & 3 deletions CONTRIBUTING.md
Expand Up @@ -10,9 +10,10 @@ Contributing to the project, especially `regexes.yaml`, is both welcomed and enc
* `tests/test_device.yaml`
* `tests/test_os.yaml`
* `tests/test_ua.yaml`
5. Push your branch to GitHub and submit a pull request
6. Monitor the pull request to make sure the Travis build succeeds. If it fails simply make the necessary changes to your branch and push it. Travis will re-test the changes.
5. Check that your regex is not vulnerable to [ReDoS](https://www.owasp.org/index.php/Regular_expression_Denial_of_Service_-_ReDoS) using the test in `tests/regexes.js`
6. Push your branch to GitHub and submit a pull request
7. Monitor the pull request to make sure the Travis build succeeds. If it fails simply make the necessary changes to your branch and push it. Travis will re-test the changes.

That's it. If you don't feel comfortable forking the project or modifying the YAML you can also [submit an issue](https://github.com/ua-parser/uap-core/issues) that includes the appropriate user agent string and the expected results of parsing.

Thanks!
Thanks!
25 changes: 24 additions & 1 deletion README.md
@@ -1,4 +1,4 @@
uap-core [![Build Status](https://secure.travis-ci.org/ua-parser/uap-core.png?branch=master)](https://travis-ci.org/ua-parser/uap-core)
uap-core [![Build Status](https://secure.travis-ci.org/ua-parser/uap-core.svg?branch=master)](https://travis-ci.org/ua-parser/uap-core) [![Backers on Open Collective](https://opencollective.com/ua-parser/backers/badge.svg)](#backers) [![Sponsors on Open Collective](https://opencollective.com/ua-parser/sponsors/badge.svg)](#sponsors)
========

This repository contains the core of [BrowserScope][2]'s original [user agent string parser][3]: data collected over the years by [Steve Souders][4] and numerous other contributors, extracted into a separate [YAML file][5] so as to be reusable _as is_ by implementations in any programming language.
Expand All @@ -8,6 +8,7 @@ This repo itself does _not_ contain a parser: only the necessary data to build o
Maintainers
-----------

* [Com Menthol](https://github.com/commenthol)
* [Lindsey Simon](https://github.com/elsigh) ([@elsigh](https://twitter.com/elsigh))
* [Tobie Langel](https://github.com/tobie) ([@tobie](https://twitter.com/tobie))

Expand All @@ -22,6 +23,28 @@ Contributing Changes to regexes.yaml

Please read the [contributors' guide](CONTRIBUTING.md)

## Credits
### Contributors

This project exists thanks to all the people who contribute. [[Contribute](CONTRIBUTING.md)].
<img src="https://opencollective.com/ua-parser/contributors.svg?width=890&button=false" />


### Backers

Thank you to all our backers! 🙏 [[Become a backer](https://opencollective.com/ua-parser#backer)]

<a href="https://opencollective.com/ua-parser#backers" target="_blank"><img src="https://opencollective.com/ua-parser/backers.svg?width=890"></a>


### Sponsors

Support this project by becoming a sponsor. Your logo will show up here with a link to your website. [[Become a sponsor](https://opencollective.com/ua-parser#sponsor)]

<a href="https://opencollective.com/ua-parser/sponsor/0/website" target="_blank"><img src="https://opencollective.com/ua-parser/sponsor/0/avatar.svg"></a>



License
-------

Expand Down
9 changes: 5 additions & 4 deletions package.json
@@ -1,7 +1,7 @@
{
"name": "uap-core",
"description": "The regex file necessary to build language ports of Browserscope's user agent parser.",
"version": "0.5.0",
"version": "0.6.8",
"maintainers": [
{
"name": "Tobie Langel",
Expand All @@ -25,11 +25,12 @@
}
],
"devDependencies": {
"yamlparser": ">=0.0.2",
"mocha": "*",
"uap-ref-impl": "*"
"safe-regex": "^2.0.1",
"uap-ref-impl": "git+https://github.com/ua-parser/uap-ref-impl#master",
"yamlparser": ">=0.0.2"
},
"scripts": {
"test": "mocha -u tdd -R min ./tests/test.js"
"test": "mocha --opts ./tests/mocha.opts ./tests"
}
}
1,147 changes: 910 additions & 237 deletions regexes.yaml

Large diffs are not rendered by default.

0 comments on commit bfdad6a

Please sign in to comment.