Tokenizes the SPDX license string (or any simarly formatted string)
into parts. Any license parts of the string e.g. "LGPL-2.0+" are prefexed
with "@", the conjunctive replaced with "&", the disjunctive replaced
with "|" and the WITH operator for license exceptions replaced with "^".
Brackets are added as indervidual tokens and other strings are
appended into single tokens where possible.
Parameters
license: string
a license string, e.g. "LGPLv2+ and (QPL or GPLv2) and MIT"
Tokenizes the SPDX license string (or any simarly formatted string) into parts. Any license parts of the string e.g. "LGPL-2.0+" are prefexed with "@", the conjunctive replaced with "&", the disjunctive replaced with "|" and the WITH operator for license exceptions replaced with "^". Brackets are added as indervidual tokens and other strings are appended into single tokens where possible.