User:Kim/Stations, Skills, Resources/Git API

scheming around with the Git API
Resources:

• GitLab Rest Api
• Gitlab Commits Api

can I use this to transform my personal reader into a (print / web to print) publication?

Issues

to mess around with my xp reader repository I created a new branch but this messed up all the curl requests so I decided to make a copy (not exactly a fork) using this command git push https://gitlab.com/km_kt/xp-reader-fork +gitpub:main (where xp reader fork is my newly initialized repo where i copy the branch gitpub from my old repo into and make it main) [1]
this actually worked! I could then simply clone it to local with http

• markdown formatting (add a class or paragraph etc so it can be selected with css)[2] -> found my way around that with manually creating html tags and pre/ appending them to the markdown file in the 2nd sh script
• weasyprint character encoding error: footnote backlinks (try other tool? try to escape characters?) >> set .footnote-back to display: none; (print does not need backlinks anyway, if they are necessary, add ::after elem with content:)

Links

js

scrape content from dynamic websites uses python (and a lot of libraries >> is there a way around that?)

• wget and curl dont run js (on a webpage) by themselves, this needs to be emulated by a pseudo browser (which is what they do above) using Selenium another option seems to be phantom.js as described here

cli / other

• cat jp https://www.baeldung.com/linux/jq-command-json

terminal

these worked:
curl --header "PRIVATE-TOKEN: <your_personal_access_token>" https://gitlab.com/api/v4/projects
curl --header "PRIVATE-TOKEN: <your_personal_access_token>" https://gitlab.com/api/v4/user
curl --header "PRIVATE-TOKEN: <your_personal_access_token>" https://gitlab.com/api/v4/projects/69537327/repository/files/queue.md/blame?ref=main (this returns file content and commit information [3])

curl --header "PRIVATE-TOKEN: glpat-D2WBV-9pNPVVfPyaOECS8m86MQp1Omc5b242Cw.01.121amvsln" "https://gitlab.com/api/v4/projects/69537327/repository/commits?path=queue.md&with_stats=true" -o output.json (this returns commits for a specific file in json format)

jq

curl --header "PRIVATE-TOKEN: glpat-D2WBV-9pNPVVfPyaOECS8m86MQp1Omc5b242Cw.01.121amvsln" "https://gitlab.com/api/v4/projects/69537327/repository/commits?path=queue.md&with_stats=true" | jq '.[] | .message' this returns contents of message as strings

| jq -s '[ .[] | map(.) | .[] | {message: .message, created: .created_at}]' >> out.json (-s (for slurp) allows JQ to process the file as a single array object)

jq -r outputs raw text without " " like here | jq -r '.[] | .message,.created_at' >> out.txt

to print

similar steps have been done here Terms of servers

pandoc

this should get all files and subdirectories recursively
**/*.md -o gitpub.html

web (js)

this works for a simple fetch request in js (important are the origin: '*' to prevent CORS)

const url = 'https://gitlab.com/api/v4/projects/69537327/repository/commits';
fetch(url, {
    method: 'GET',
    origin: '*',
    headers: {
        'PRIVATE-TOKEN': 'glpat-D2WBV-9pNPVVfPyaOECS8m86MQp1Omc5b242Cw.01.121amvsln',
        'Content-Type': 'application/json',
     },
})

This Example fetches Commits for the file queue.md: https://hub.xpub.nl/cerealbox/~kim/test-3.html
Problem: this is a dynamic site, when trying to convert its contents (for printing to .md), nothing is there
-> tried Curl solution (see above), managed to curl same contents into json - now pandoc error when converting json into .md
pandoc -f csljson -t gfm queue-commits.json -o queue-commits.md this is the command i tried out