hikr.org-backup - Sicherungswerkzeug / Backup Tool


Publiziert von carpintero, 17. November 2014 um 23:24. Diese Seite wurde 1534 mal angezeigt.

Bitte beachten
Der Inhalt von diesem Forumbeitrag wird nicht mehr aktualisiert. Aktuelle Informationen befinden sich auf der Seite https://github.com/simongysi/hikr-ruby.
 
Please note
The content of this forum thread will no longer be updated. Please find the most recent information on the site https://github.com/simongysi/hikr-ruby.

29.04.2018


Please find the english version below...
 
Deutsch
Ich habe für mich ein Skript entwickelt, um meine Berichte auf Hikr.org zu sichern. Da dieses Thema in der Vergangenheit auch im Forum aufgegriffen wurde und offenbar ein Bedürfnis dafür besteht, stelle ich das Skript zur freien Verfügung. Eine Liste mit den Merkmalen und bekannten Problemen bzw. Einschränkungen, sowie eine Anleitung zur Installation befindet sich im Anschluss.
 
Hinweis:
Das Ermitteln des Journals kann bei vielen Berichten mehrere Minuten dauern (keine Fortschrittsanzeige).
 
Merkmale:
  1. Erstellt eine Text Datei mit den wichtigsten Informationen des Berichts
  2. Lädt alle Dateien im Geodata Abschnitt herunter
  3. Lädt alle Bilder in ihrer original Grösse herunter
  4. Erzeugt ein PDF Dokument der Druckansicht des Berichts
  5. Erzeugt eine ZIP Datei mit diesen Daten für jeden Bericht
  6. Verfügbar für Windows, Linux und OS X
  7. Unterstützt mehrere Sprachen (English, Deutsch, Italiano, Français, Español)
  8. Erstellt eine Text- und eine CSV Datei des Journals
 
Installation:
  1. Öffne https://github.com/zitronenfalter/hikr.org-backup
  2. Folge den Anweisungen im README.md Abschnitt der Webseite
  3. Klick Download ZIP (auf der rechten Seite)
  4. Extrahiere das ZIP Archiv in ein Verzeichnis deiner Wahl
  5. Benutze ruby hikr.org-backup.rb --help um mehr Informationen zu erhalten
 
Bekannte Probleme:
  1. Routen (angehängt an Wegpunkte) werden nicht in die Sicherungskopie einbezogen
  2. Kommentare werden nicht in die Sicherungskopie einbezogen
  3. Elemente mit einem Symbol in der Galerie werden nicht in die Sicherungskopie einbezogen, z.B. Quicktime Filme, PDF Dokumente...
  4. Die Sicherungskopie schlägt fehl, falls ein Bild nicht in seiner original Grösse vorhanden ist
  5. Das erzeugte PDF Dokument weist kleinere Unschönheiten in der Darstellung auf
 
English
I've developed a script to create backups of my reports at Hikr.org for my own use. I decided to make the script available for anyone, because this topic was already discussed in the forum and seems to be a need. You'll find the features and known issues listed below, as well as a description of the installation procedure.
 
Remark:
Retrieving the journal may take several minutes if there are many reports (no progress bar).
 
Features:
  1. Creates a text file with the relevant information of the report
  2. Downloads all files of the geodata section
  3. Downloads all images of the gallery in their original size
  4. Generates a PDF document of the print layout of the report
  5. Creates a ZIP file of these data for each report
  6. Available for Windows, Linux and OS X
  7. Supports multiple languages (English, Deutsch, Italiano, Français, Español)
  8. Creates a text- and a CSV file of the journal
 
Installation:
  1. Open https://github.com/zitronenfalter/hikr.org-backup
  2. Follow the instructions in the README.md section of the website
  3. Click Download ZIP (on the right side)
  4. Extract the ZIP archive to a directory of your choice
  5. Use ruby ruby hikr.org-backup.rb --help to get more information
 
Known issues:
  1. Routes (attached to waypoints) are not included in the backup
  2. Comments of reports or images are not included in the backup
  3. Items with an icon in the gallery are not included in the backup, e.g. Quicktime movies, PDF documents...
  4. The backup fails, if an image is not available in its original size
  5. The generated PDF document suffers from minor rendering problems


Edited on 01.12.2014:
  • News added
  • Minor corrections
  • Change history
    • Added feature that saves the journal as text- and CSV file
    • Overhauled README.md

Edited on 20.11.2014:
  • Added OS X to the supported platforms
  • Added supported languages



Kommentare (24)


Kommentar hinzufügen

felixbavaria hat gesagt:
Gesendet am 18. November 2014 um 11:31
That seems like a very helpful tool, thanks for posting! I will try it on my home PC tonight.

I just hope people will use it in a reasonable way (i.e., only to backup their own reports or selected other reports), in order not to create too much traffic ...

Felix

gbal hat gesagt: Hikr Backup: my personal thought
Gesendet am 19. November 2014 um 12:51
Could be many reasons to wish to backup your own reports on Hikr. You can keep in somewhere in your PC the documents, the pictures and other data you used to build your report; so you can think you are able to rebuild the report in case of Hikr servers failure, virus infection or other damages.
This is true but is not enough; what you lost is the Hikr page structure and design, the comments and the way to move through the pictures.
Some year ago I surfed the Web looking for some program helping to backup my reports in a better way. I found that WinHTTrack was the best program for this purpose; I saw that is a horse difficult to ride because you have to set many parameters to say it how to do the work but my son Marcello helped me and we gathered a satisfying parameters collection. After the initial effort you can always use those settings every time you backup a new report. Referring to the “Known issues” listed above WinTTrack solves all the points except the Minimap (Route) that is not shown if Hikr site is not reachable. But you can “save” your Route if it is listed under the Geodata and use it with other programs. You can use perfectly your backup without Internet connection and without Hikr server.
All the comments are kept, no matter if they are on the main page or under a picture.
All the picture caption you wrote are kept as well as the PDF documents if they are present.
The labels are in English even if on the original report you set another language; i.e. if you read “Data della gita:” on the original report you will read “Date of the hike:” on the backed up one but this not seems to be a problem.

The WinTTrack program is downloadable free of charge from the site:
http://www.httrack.com/page/2/en/index.html where you can choose the right product for you.

Below I list the parameters collection I use for my Hikr backup to give you a way ready to use (please note you have to exactly write the following parameters for the best result):

Limits:
Mirroring Depth: 3
Max Transfer Rate: 100000 B/s

Scan Rules:
Filters:
+*.png +*.gif +*.jpg +*.css +*.js -ad.doubleclick.net/* -mime:application/foobar
-*
-www.hikr.org/gallery/
-www.hikr.org/gallery/*
+www.hikr.org/gallery/photo*
+f.hikr.org/files/*.gpx
+f.hikr.org/files/*.kml
+f.hikr.org/files/*[0-9]s.jpg
+f.hikr.org/files/*[0-9].jpg
+f.hikr.org/files/*[0-9]l.jpg
+f.hikr.org/files/*[0-9].mov
+f.hikr.org/files/*[0-9].pdf
+s.hikr.org/*
-www.hikr.org/gallery/portal.html
-www.hikr.org/gallery/?mode=*
-www.hikr.org/gallery/tags/*
-translate.google.com/*
-f.hikr.org/files/
-www.hikr.org/tour/*
-www.hikr.org/gallery/?*
-*.flagcounter.com/*
-*.youtube.com/*


Italiano
Possono esserci molte ragioni per desiderare di fare un backup personale dei propri rapporti su Hikr. Voi potete conservare da qualche parte del vs. PC i documenti, le foto e gli altri dati che avete usato per costruire il vs. rapporto; in questo modo potreste pensare di essere in grado di ricostruire il report in caso di guasti ai server di Hikr, alla penetrazione di qualche virus o di qualche altro danno. Questo è vero ma non è abbastanza; ciò che perdereste è la struttura ed il design della pagina di Hikr, i commenti e il meccanismo per passare da una foto all’altra.
Qualche anno fa ho girato un po’ sul Web cercando un programma che potesse fare meglio il backup dei miei report. Ho trovato che WinHTTrack era quanto di meglio per questo scopo; ho visto però che non era facile cavalcare questo cavallo perchè dovevi caratterizzare molti parametri per dire al programma come fare il lavoro ma con l’aiuto di mio figlio Marcello abbiamo raccolto la giusta collezione di parametri. Dopo lo sforzo iniziale questi parametri vengono usati allo stesso modo ogni volta che si fa il backup di un nuovo rapporto.
Riferendomi ai “Problemi conosciuti” elencati sopra WinHTTrack li risolve tutti eccetto la Minimap che non viene mostrata se manca la connessione al sito di Hikr. Ma potete salvare la vs. Traccia se ne avete elencata una sotto Geodata e usarla con altri programmi. Per tutto il resto potete usare i vs. backup perfettamente offline, senza connessione internet nè connessione ad Hikr.
Vengono mantenuti tutti i commenti sia sulla pagina pricipale che quelli posti sotto la singola foto. Vengono inoltre mantenute le didascalie che avete scritto per le foto come pure eventuali documenti PDF. Le etichette del report salvato sono in Inglese anche se quando lo avete scritto erano in un’altra lingua; per intenderci se sul report originale leggevate: “Data della gita:” sul report salvato leggerete “Date of the hike:” ma questo non è un un problema.

Il programma WinHTTrack è scaricabile gratuitamente qui:
http://www.httrack.com/page/2/en/index.html dove troverete il prodotto che meglio si confà a voi.

Qui sotto elenco il set di parametri che uso per fare Hikr backup in modo da facilitarvi il compito (mi raccomando di scrivere i parametri esattamente come sono elencati).

Limits:
Mirroring Depth: 3
Max Transfer Rate: 100000 B/s

Scan Rules:
Filters:
+*.png +*.gif +*.jpg +*.css +*.js -ad.doubleclick.net/* -mime:application/foobar
-*
-www.hikr.org/gallery/
-www.hikr.org/gallery/*
+www.hikr.org/gallery/photo*
+f.hikr.org/files/*.gpx
+f.hikr.org/files/*.kml
+f.hikr.org/files/*[0-9]s.jpg
+f.hikr.org/files/*[0-9].jpg
+f.hikr.org/files/*[0-9]l.jpg
+f.hikr.org/files/*[0-9].mov
+f.hikr.org/files/*[0-9].pdf
+s.hikr.org/*
-www.hikr.org/gallery/portal.html
-www.hikr.org/gallery/?mode=*
-www.hikr.org/gallery/tags/*
-translate.google.com/*
-f.hikr.org/files/
-www.hikr.org/tour/*
-www.hikr.org/gallery/?*
-*.flagcounter.com/*
-*.youtube.com/*


83_Stefan hat gesagt: RE:Hikr Backup: my personal thought
Gesendet am 19. November 2014 um 16:47
That sounds very interesting! I think, I'll give it a try next time when snow makes a trip unpossible... best regards and thanks a million!

gbal hat gesagt: RE:Hikr Backup: my personal thought
Gesendet am 19. November 2014 um 16:55
I'm happy to give you an interesting work to do next winter Stefan.
Bye
Giulio

giorgio59m hat gesagt: RE:Hikr Backup: my personal thought
Gesendet am 19. November 2014 um 17:20
Well done, Giulio !
;)

gbal hat gesagt: RE:Hikr Backup: my personal thought
Gesendet am 19. November 2014 um 17:26
Thanks Giorgio

Gelöschter Kommentar

gbal hat gesagt: RE:
Gesendet am 19. November 2014 um 18:47
Thank you very much wegener.
You don't need to compile a text file with the list of your reports.
WinHTTrack already gives you a file index.html that links you to the backed up reports stored into your PC.
Please see here: http://www.hikr.org/post88269.html

Gelöschter Kommentar

gbal hat gesagt: RE:
Gesendet am 19. November 2014 um 19:04
No. I backup a single report at a time.
It's useful to wait a couple a weeks from the publishing waiting for the comments; then I backup that report, only.
The backup operation is not so fast then I think to do an user backup would require a very long time.

carpintero Pro hat gesagt: RE:Hikr Backup: my personal thought
Gesendet am 19. November 2014 um 19:49
Hi gbal

Thanks for your comment and sharing the settings of WinHTTrack. I d'like to add some comments... :)

> Referring to the “Known issues” listed above WinTTrack solves all the points except the Minimap (Route) that is not shown if Hikr site is not reachable. But you can “save” your Route if it is listed under the Geodata and use it with other programs.

Just to clarify... the first point of the "Known issues" is refering to routes attached to waypoints - not the minimap. You'll find an example in the report *Chli Chärpf (2700m) – Hitzetour durchs Glarnerland where a route is attached to the waypoint  Chli Kärpf.

> The labels are in English even if on the original report you set another language; i.e. if you read “Data della gita:” on the original report you will read “Date of the hike:” on the backed up one but this not seems to be a problem.

You can specify the requested language on the "Browser ID" tab (in the same dialog as "Scan Rules"). Edit the value in the "Language" field - default is "en,*" (without quotes). Some of the possible values are "de", "en", "it", "fr", "es"... (of course, without quotes too).

I may mention here, that the hikr.org-backup script is supporting different languages too (the same way).

Best regards
Simon

gbal hat gesagt: RE:Hikr Backup: my personal thought
Gesendet am 20. November 2014 um 10:15
Hi Simon,
thank you for first opening this subject; you explained your method then I added mine. I hope that the Hikrs now think about this important thing and decide how to behave. Thank you for the last suggestions and clarifications too.
Have a nice day
Giulio
Giulio

83_Stefan hat gesagt:
Gesendet am 19. November 2014 um 16:42
Hallo carpintero, vielen Dank, dass du dir diese Mühe gemacht hast und dein Ergbnis mit allen teilst. Das spricht für dich. Beste Grüße!

Bergpoetin hat gesagt: Nice
Gesendet am 20. November 2014 um 11:23
But what about MacOS?

carpintero Pro hat gesagt: RE:Nice
Gesendet am 20. November 2014 um 13:53
As far as i know, all the dependencies (ruby+gems, wkhtmltopdf) are available for OSX. It is possible, that the script works for Mac too (in fact, i expect it to work), but i didn't test it until now. Unfortuntely i don't have the possibility to test it by myself, because i don't have a Mac and it is not easy to virtualize OSX.

SCM hat gesagt: RE:Nice
Gesendet am 20. November 2014 um 17:10
I tested it with OS X Yosemite and it worked fine. I used homebrew to install all the dependencies. The only problem I encountered was the installation of nokogiri which seems to be a common problem. This post helped me a lot: http://stackoverflow.com/questions/5528839/why-does-installing-nokogiri-on-mac-os-fail-with-libiconv-is-missing?rq=1

carpintero Pro hat gesagt: RE:Nice
Gesendet am 20. November 2014 um 21:39
Wow, thank you very much for testing this!

SCM hat gesagt: RE:Nice
Gesendet am 21. November 2014 um 11:17
No problem. So you can add the OSX installation to your readme file. If you need help, I can provide you with the exact terminal commands i used.

MatthiasG hat gesagt: fail
Gesendet am 1. Dezember 2014 um 15:58
I have tried to install this on a windows machine without direct internet connection and failed to install nokogiri. First it fails to download zlib. Using the parameter --use-system-libraries it then tells me I am missing libxml2, but when I try to install this I get the missing zlib again, even though it seems to recognize the --use-system-libraries parameter...

Giving up now. I am a software developer and this took me half an hour so far. I think non-IT-people can save themselves the trouble to even try installing this without open internet connection. It would really be nice to have an offline installer for this.

carpintero Pro hat gesagt: RE:fail
Gesendet am 3. Dezember 2014 um 20:33
Hi Matthias

nokogiri seems to be a hard to install (on OS X as well). If online, usually it's no problem installing it on Windows - even no build environment is required, because prebuilt binaries are included in the package.

Unfortunately, this depends also on Ruby version you have installed on your machine. At least for now, i recommend using Ruby 2.0.0-p598 (or the 64-bit variant), downloadable from http://rubyinstaller.org/.

Maybe i will replace nokogiri by an alternative in the future.

Thanks for feedback anyway,
Simon

MatthiasG hat gesagt: RE:fail
Gesendet am 4. Dezember 2014 um 15:18
Thanks for the info! I'll give it another try then.

carpintero Pro hat gesagt: RE:fail
Gesendet am 4. Dezember 2014 um 21:13
It is a bit cumbersome with this Ruby stuff... with the Ruby version above, it will give you most likely a SSL certificate error if you're trying to install Ruby Gems. Changing the Gems sources to the non-HTTPS worked for me (of course at the cost of the security risk):

> gem sources -r https://rubygems.org/
> gem sources -a http://rubygems.org/

carpintero Pro hat gesagt: RE:fail
Gesendet am 5. Dezember 2014 um 11:06
I learn everyday something new... :)

Today it's again about the version of Ruby on Windows.
I strongly recommend to use Ruby 1.9.3, at least on Windows. This version is also recommended by the site http://rubyinstaller.org/ and seems to work without troubles (no problems with nokogiri, no troubles with SSL certificate error).

I will update the README.md as soon as possible.

carpintero Pro hat gesagt: Neues Merkmal / New Feature
Gesendet am 1. Dezember 2014 um 20:21
NEU: Erstellt eine Text- und eine CSV Datei des Journals
NEW: Creates a text- and a CSV file of the journal


Kommentar hinzufügen»