Did you ever need to write XML parser from scratch? You can have a parser ready in few minutes! Let me introduce you to xsd2go.

Why bother?

Most of my readers will probably have an experience with the wide spread XML applications like RSS or Atom feeds, SVG, XHTML. For those well known XML applications you will find good library encapsulating the parsing for you. You just include existing parser in your project and you are done with it. However, what would you do if you cannot use it (think of license mismatch), or what would you do if there was no parsing library at all?

There are many XML applications around. Here comes a (probably incomplete) list of XML formats, I had to touch in my past life: Atom, DocBook, Office Open XML, OpenDocument (ODF), OSCAL, Rolie, RSS, SAML, SCAP (+dozens of sub-formats), SOAP, SVG, XMPP, Epub, WS-Policy, XHTML, XSLT.

What is XSD?

You already know that, but let me briefly run through it. XSD stands for XML Schema Definition. For a given XML application (think of Rss) it describes how a well-formed document looks like, describing the structure really well. XSD will tell you, what attributes each element has, what sub-elements can be found in each element and what are the cardinalities, meaning how many of sub-elements of certain type you can expect and which are optional and which are not.

In effect (and by design), XSD can be used to automatically assess documents adherence to a given standard.

Little snark: XSD is true XML application and hence it is expressive and you will find many ways to achieve the same descriptive result.

What is XSD2Go?

XSD2Go is minimalistic project that converts XSD to golang code. For given set of XSD files, xsd2go produces golang code that contains structures/models relevant for parsing given XML format. Produced golang code contains XML parsing hints that can be used together with standard encoding/xml golang package to parse the XMLs.

⚠️ You should run xsd2go, before ever importing encoding/xml to your project. ⚠️

I cannot stress this enough.

I mean, xsd2go is 6 days old and thus very unfinished, but I still believe it already presents good value compared to starting from scratch.

How does it work?

Just briefly. Xsd2go will

  • parse your master XSD file
    • and processes all xsd:import elements, parsing imported XSD files
    • effectively building workspace and a dependency tree of all relevant XSDs.
    • Circular dependencies are allowed.
  • pre-process each XSD
    • verify validity of internal references (think of xsd:element/@ref, xsd:attribute/@ref, @type and similar)
    • figure-out golang import () paths needed (as a reminder each one XSD file results in one golang package)
    • discover and fix any name space clashes that would result in invalid golang code
    • re-compute cardinalities of elements that are wrapped by xsd:sequence and xsd:choice
    • figure out golang types
  • produce golang code using simple and readable text/template golang package
  • re-formats created golang code using go/format

Show me an exemplary usage

gocomply_xsd2go convert schemas/sds/1.3/scap-source-data-stream_1.3.xsd github.com/gocomply/scap pkg/scap/models
Processing 'schemas/sds/1.3/scap-source-data-stream_1.3.xsd'
	Parsing: schemas/sds/1.3/scap-source-data-stream_1.3.xsd
	Parsing: schemas/xccdf/1.2/xccdf_1.2.xsd
	Parsing: schemas/common/xml.xsd
	Parsing: schemas/xccdf/1.2/cpe-language_2.3.xsd
	Parsing: schemas/cpe/2.3/cpe-naming_2.3.xsd
	Parsing: schemas/oval/5.11.3/oval-definitions-schema.xsd
	Parsing: schemas/oval/5.11.3/oval-common-schema.xsd
	Parsing: schemas/common/xmldsig-core-schema.xsd
	Parsing: schemas/oval/5.11.3/aix-definitions-schema.xsd
	Parsing: schemas/oval/5.11.3/android-definitions-schema.xsd
	Parsing: schemas/oval/5.11.3/apache-definitions-schema.xsd
	Parsing: schemas/oval/5.11.3/apple-ios-definitions-schema.xsd
	Parsing: schemas/oval/5.11.3/asa-definitions-schema.xsd
	Parsing: schemas/oval/5.11.3/catos-definitions-schema.xsd
	Parsing: schemas/oval/5.11.3/esx-definitions-schema.xsd
	Parsing: schemas/oval/5.11.3/freebsd-definitions-schema.xsd
	Parsing: schemas/oval/5.11.3/hpux-definitions-schema.xsd
	Parsing: schemas/oval/5.11.3/independent-definitions-schema.xsd
	Parsing: schemas/oval/5.11.3/ios-definitions-schema.xsd
	Parsing: schemas/oval/5.11.3/iosxe-definitions-schema.xsd
	Parsing: schemas/oval/5.11.3/junos-definitions-schema.xsd
	Parsing: schemas/oval/5.11.3/linux-definitions-schema.xsd
	Parsing: schemas/oval/5.11.3/macos-definitions-schema.xsd
	Parsing: schemas/oval/5.11.3/netconf-definitions-schema.xsd
	Parsing: schemas/oval/5.11.3/pixos-definitions-schema.xsd
	Parsing: schemas/oval/5.11.3/sharepoint-definitions-schema.xsd
	Parsing: schemas/oval/5.11.3/solaris-definitions-schema.xsd
	Parsing: schemas/oval/5.11.3/unix-definitions-schema.xsd
	Parsing: schemas/oval/5.11.3/windows-definitions-schema.xsd
	Parsing: schemas/cpe/2.3/cpe-dictionary_2.3.xsd
	Parsing: schemas/ocil/2.0/ocil-2.0.xsd
	Parsing: schemas/common/catalog.xsd
	Parsing: schemas/common/xlink.xsd
	Generating 'pkg/scap/models/ds/models.go'
	Generating 'pkg/scap/models/apple_ios_def/models.go'
	Generating 'pkg/scap/models/macos_def/models.go'
	Generating 'pkg/scap/models/cpe/models.go'
	Generating 'pkg/scap/models/android_def/models.go'
	Generating 'pkg/scap/models/catos_def/models.go'
	Generating 'pkg/scap/models/cdf/models.go'
	Generating 'pkg/scap/models/xml_dsig/models.go'
	Generating 'pkg/scap/models/aix_def/models.go'
	Generating 'pkg/scap/models/apache_def/models.go'
	Generating 'pkg/scap/models/hpux_def/models.go'
	Generating 'pkg/scap/models/ind_def/models.go'
	Generating 'pkg/scap/models/xlink/models.go'
	Generating 'pkg/scap/models/oval/models.go'
	Generating 'pkg/scap/models/esx_def/models.go'
	Generating 'pkg/scap/models/sp_def/models.go'
	Generating 'pkg/scap/models/sol_def/models.go'
	Generating 'pkg/scap/models/cpe_dict/models.go'
	Generating 'pkg/scap/models/inter/models.go'
	Generating 'pkg/scap/models/oval_def/models.go'
	Generating 'pkg/scap/models/asa_def/models.go'
	Generating 'pkg/scap/models/junos_def/models.go'
	Generating 'pkg/scap/models/pixos_def/models.go'
	Generating 'pkg/scap/models/iosxe_def/models.go'
	Generating 'pkg/scap/models/linux_def/models.go'
	Generating 'pkg/scap/models/netconf_def/models.go'
	Generating 'pkg/scap/models/unix_def/models.go'
	Generating 'pkg/scap/models/er/models.go'
	Generating 'pkg/scap/models/freebsd_def/models.go'
	Generating 'pkg/scap/models/ios_def/models.go'
	Generating 'pkg/scap/models/win_def/models.go'

The outputs can be found at github.com/GoComply/scap/pkg/scap/model

and lastly, kudos

… go to Gabe Alford who kept nudging me about the need for such project. Thank You, Gabe!