User:Kim/Stations, Skills, Resources/Git API

scheming around with the Git API
Resources:

• GitLab Rest Api
• Gitlab Commits Api

can I use this to transform my personal reader into a (print / web to print) publication?

Links

js

scrape content from dynamic websites uses python (and a lot of libraries >> is there a way around that?)

• wget and curl dont run js (on a webpage) by themselves, this needs to be emulated by a pseudo browser (which is what they do above) using Selenium another option seems to be phantom.js as described here

cli / other

• cat jp https://www.baeldung.com/linux/jq-command-json

terminal

these worked:
curl --header "PRIVATE-TOKEN: <your_personal_access_token>" https://gitlab.com/api/v4/projects
curl --header "PRIVATE-TOKEN: <your_personal_access_token>" https://gitlab.com/api/v4/user
curl --header "PRIVATE-TOKEN: <your_personal_access_token>" https://gitlab.com/api/v4/projects/69537327/repository/files/queue.md/blame?ref=main (this returns file content and commit information [1])

curl --header "PRIVATE-TOKEN: glpat-D2WBV-9pNPVVfPyaOECS8m86MQp1Omc5b242Cw.01.121amvsln" "https://gitlab.com/api/v4/projects/69537327/repository/commits?path=queue.md&with_stats=true" -o output.json (this returns commits for a specific file in json format)

jq

curl --header "PRIVATE-TOKEN: glpat-D2WBV-9pNPVVfPyaOECS8m86MQp1Omc5b242Cw.01.121amvsln" "https://gitlab.com/api/v4/projects/69537327/repository/commits?path=queue.md&with_stats=true" | jq '.[] | .message' this returns contents of message as strings

| jq -s '[ .[] | map(.) | .[] | {message: .message, created: .created_at}]' >> out.json (-s (for slurp) allows JQ to process the file as a single array object)

jq -r outputs raw text without " " like here | jq -r '.[] | .message,.created_at' >> out.txt

web (js)

this works for a simple fetch request in js (important are the origin: '*' to prevent CORS)

const url = 'https://gitlab.com/api/v4/projects/69537327/repository/commits';
fetch(url, {
    method: 'GET',
    origin: '*',
    headers: {
        'PRIVATE-TOKEN': 'glpat-D2WBV-9pNPVVfPyaOECS8m86MQp1Omc5b242Cw.01.121amvsln',
        'Content-Type': 'application/json',
     },
})

This Example fetches Commits for the file queue.md: https://hub.xpub.nl/cerealbox/~kim/test-3.html
Problem: this is a dynamic site, when trying to convert its contents (for printing to .md), nothing is there
-> tried Curl solution (see above), managed to curl same contents into json - now pandoc error when converting json into .md
pandoc -f csljson -t gfm queue-commits.json -o queue-commits.md this is the command i tried out