KnowYourCG

Install — then fetch once

# build from source (YAME is a submodule)
git clone --recurse-submodules \
  https://github.com/zhou-lab/kycg
cd kycg && make

Dependencies are YAME's — vendored htslib, zlib, libm, libcurl — with YAME linked statically from a pinned submodule. kycg --version reports the coupled YAME version.

# guided: location, collection, sets, confirm
kycg fetch

# or name what you want outright
kycg fetch mm10
kycg fetch hg38:CGI,ChromHMM

# open with hg38 checked; f to start
kycg fetch hg38

The store — fetch, verified

Knowledgebases lived in three places with three access patterns and no verification. fetch is the tie.

# -f skips the browser: plan, then fetch
$ kycg fetch -f mm10:CGI

==> Knowledgebase sets to download
    mm10  •  KYCGKB_mm10 v2  •  ~/.local/share/yame/mm10/KYCG

    CGI.20220904.cm                                  120 KB
    cpg_nocontig.cr                                 22.7 MB

    2 file(s), 22.8 MB

✓ CGI.20220904.cm                    120 KB
✓ cpg_nocontig.cr                    22.7 MB

✓ 2 fetched (22.8 MB).

…into a plain layout keyed on the target you fetch — the address you browse to, not the repo it came from — shared with the rest of the suite. A .cm in the store is an ordinary file that kycg test -m takes by path. No database, no lockfile, no bookkeeping.

~/.local/share/yame/
├ mm10/                   # whole genome (KYCGKB_mm10)
│ ├ cpg_nocontig.cr         # the row list
│ └ KYCG/
│   ├ ChromHMM.20220414.cm
│   └ SHA256SUMS
└ MSA/                    # array (InfiniumAnnotation)
  ├ MSA.ordering.tsv.gz     # probe ordering
  ├ SHA256SUMS              # platform manifest
  └ KYCG/
    ├ CGI.20220904.cm(+.idx)
    └ SHA256SUMS

Pick what you need — r, i, -o

The full hg38 collection is 363 MB. You rarely want all of it.

# a tree — → unfolds a target in place
$ kycg fetch
    target  kind          rows        source           cached_sets
  ▸ hg38    whole genome  29,401,795  KYCGKB_hg38 v2   3/32
❯ ▾ mm10    whole genome  21,867,837  KYCGKB_mm10 v2  28/28
    ├ CGI       CGI…cm       120 KB  cached
    ├ ChromHMM  ChromHMM…cm  857 KB  cached
    ├ PMD       PMD…cm      16.3 KB  cached
  row 1 of 9 · → open  space select  r recommended  f fetch  i info  d store

# or name it directly — opens with those checked
$ kycg fetch hg38:CGI,ChromHMM

# no browser, no questions (scripts, Docker, Nextflow)
$ kycg fetch -f hg38:CGI,ChromHMM

r checks the recommended selection for that collection — the dozen sets worth having before you know what you are looking for, instead of all forty. i opens a pane answering what a set actually is. It is on by default and follows the cursor, so arrowing down walks you through the catalogue with each set explained as you reach it — i hides it when you want the full screen for scanning:

  TFBSrm  Transcription factor binding sites, ReMap

  A larger, uniformly reprocessed TF binding compendium — around 1,188
  factors — built from published ChIP-seq rather than one lab's panel.
  The rm suffix is ReMap, not repeat-masked.

  source      ReMap 2022, non-redundant MACS2 peaks
  citation    Hammal et al. 2022, Nucleic Acids Res, doi:10.1093/nar/gkab996
  processing  remap2022_nr_macs2 peaks, one set per factor.

Test — the core

Hypergeometric tail, six effect sizes, stratified FDR. 21.8M rows in 0.12 s.

kycg test -m ~/.local/share/yame/mm10/KYCG/ChromHMM.20220414.cm \
  query.cg > res.tsv

-m is repeatable — this is what makes a store worth having. One pass, one table, one FDR correction:

kycg test $(for f in ~/.local/share/yame/mm10/KYCG/*.cm; do
        printf -- '-m %s ' $f; done) \
  query.cg > res.tsv

Omit -m on a terminal and kycg offers the store — filtered to the sets that share your query's row space, so the list cannot be picked wrong:

$ kycg test onecell.cg

Knowledgebases for 21,867,837 rows -- space to choose, t to test
    target  kind          rows        cached_sets
❯ ▾ mm10    whole genome  21,867,837  28
   ├ [ ] Blacklist              Blacklist.20220304.cm            cached
   ├ [x] CGI                    CGI.20220904.cm                  cached
   ├ [x] ChromHMM               ChromHMM.20220414.cm             cached
   ├ [ ] ChromHMMfullStack      ChromHMMfullStack…cm             cached
  …
  → open  space select  r recommended  i hide  f fetch  t test

It is the same browser fetch uses, showing only the collections whose row count matches the query. A set you do not have yet is still listed: check it, f to fetch it, then t to test against it without leaving. Both fall back to plain numbered text under NO_COLOR or TERM=dumb, and fetch writes TSV whenever stdout is redirected.

Read the result — top hits

One row per (query sample, knowledgebase record), sorted by significance.

$ kycg test -m ChromHMM.cm -m CGI.cm -m PMD.cm \
    onecell.cg | cut -f3,4,10,13

db_file          db         estimate  fdr
PMD…cm           commonHMD  0.275     0
ChromHMM…cm      Quies      0.199     0
ChromHMM…cm      QuiesG     0.253     0
ChromHMM…cm      Tx         0.307     0
CGI…cm           Shore      0.196     9.17e-266

Check the row space — yame info

Everything is positional: row i must mean the same CpG in the query and in the knowledgebase. Confirm with yame info (kycg's backend) before you test.

$ yame info onecell.cg ChromHMM.20220414.cm
File        Sample Ncol Nrow     Format UnitBytes Keys
onecell.cg  1      NA   21867837 3      1         NA
ChromHMM…cm 1      NA   21867837 2      1         N=19|Quies,…

Mismatches fail loudly, naming both files and both counts:

Row count mismatch: query 'small.cg' record '1'
has 5 rows but knowledgebase 'chromhmm.cm'
record '1' has 21867837 rows.
These files index different reference row
lists and cannot be compared.

Plot — cinderplot, separately

kycg emits TSV; cinderplot renders it as its own process. No linking, so Cairo never enters kycg's dependency surface.

# volcano
cinderplot 'res.tsv
  + aes(estimate, neglog10_fdr)
  + geom_point() + theme_minimal()' -o v.pdf