hikr.org-backup - Sicherungswerkzeug / Backup Tool


Published by carpintero, 17 November 2014, 23h24. This page has been displayed 1643 times.

Bitte beachten
Der Inhalt von diesem Forumbeitrag wird nicht mehr aktualisiert. Aktuelle Informationen befinden sich auf der Seite https://github.com/simongysi/hikr-ruby.
 
Please note
The content of this forum thread will no longer be updated. Please find the most recent information on the site https://github.com/simongysi/hikr-ruby.

29.04.2018


Please find the english version below...
 
Deutsch
Ich habe für mich ein Skript entwickelt, um meine Berichte auf Hikr.org zu sichern. Da dieses Thema in der Vergangenheit auch im Forum aufgegriffen wurde und offenbar ein Bedürfnis dafür besteht, stelle ich das Skript zur freien Verfügung. Eine Liste mit den Merkmalen und bekannten Problemen bzw. Einschränkungen, sowie eine Anleitung zur Installation befindet sich im Anschluss.
 
Hinweis:
Das Ermitteln des Journals kann bei vielen Berichten mehrere Minuten dauern (keine Fortschrittsanzeige).
 
Merkmale:
  1. Erstellt eine Text Datei mit den wichtigsten Informationen des Berichts
  2. Lädt alle Dateien im Geodata Abschnitt herunter
  3. Lädt alle Bilder in ihrer original Grösse herunter
  4. Erzeugt ein PDF Dokument der Druckansicht des Berichts
  5. Erzeugt eine ZIP Datei mit diesen Daten für jeden Bericht
  6. Verfügbar für Windows, Linux und OS X
  7. Unterstützt mehrere Sprachen (English, Deutsch, Italiano, Français, Español)
  8. Erstellt eine Text- und eine CSV Datei des Journals
 
Installation:
  1. Öffne https://github.com/zitronenfalter/hikr.org-backup
  2. Folge den Anweisungen im README.md Abschnitt der Webseite
  3. Klick Download ZIP (auf der rechten Seite)
  4. Extrahiere das ZIP Archiv in ein Verzeichnis deiner Wahl
  5. Benutze ruby hikr.org-backup.rb --help um mehr Informationen zu erhalten
 
Bekannte Probleme:
  1. Routen (angehängt an Wegpunkte) werden nicht in die Sicherungskopie einbezogen
  2. Kommentare werden nicht in die Sicherungskopie einbezogen
  3. Elemente mit einem Symbol in der Galerie werden nicht in die Sicherungskopie einbezogen, z.B. Quicktime Filme, PDF Dokumente...
  4. Die Sicherungskopie schlägt fehl, falls ein Bild nicht in seiner original Grösse vorhanden ist
  5. Das erzeugte PDF Dokument weist kleinere Unschönheiten in der Darstellung auf
 
English
I've developed a script to create backups of my reports at Hikr.org for my own use. I decided to make the script available for anyone, because this topic was already discussed in the forum and seems to be a need. You'll find the features and known issues listed below, as well as a description of the installation procedure.
 
Remark:
Retrieving the journal may take several minutes if there are many reports (no progress bar).
 
Features:
  1. Creates a text file with the relevant information of the report
  2. Downloads all files of the geodata section
  3. Downloads all images of the gallery in their original size
  4. Generates a PDF document of the print layout of the report
  5. Creates a ZIP file of these data for each report
  6. Available for Windows, Linux and OS X
  7. Supports multiple languages (English, Deutsch, Italiano, Français, Español)
  8. Creates a text- and a CSV file of the journal
 
Installation:
  1. Open https://github.com/zitronenfalter/hikr.org-backup
  2. Follow the instructions in the README.md section of the website
  3. Click Download ZIP (on the right side)
  4. Extract the ZIP archive to a directory of your choice
  5. Use ruby ruby hikr.org-backup.rb --help to get more information
 
Known issues:
  1. Routes (attached to waypoints) are not included in the backup
  2. Comments of reports or images are not included in the backup
  3. Items with an icon in the gallery are not included in the backup, e.g. Quicktime movies, PDF documents...
  4. The backup fails, if an image is not available in its original size
  5. The generated PDF document suffers from minor rendering problems


Edited on 01.12.2014:
  • News added
  • Minor corrections
  • Change history
    • Added feature that saves the journal as text- and CSV file
    • Overhauled README.md

Edited on 20.11.2014:
  • Added OS X to the supported platforms
  • Added supported languages



Comments (24)


Post a comment

Sent 18 November 2014, 11h31
That seems like a very helpful tool, thanks for posting! I will try it on my home PC tonight.

I just hope people will use it in a reasonable way (i.e., only to backup their own reports or selected other reports), in order not to create too much traffic ...

Felix

gbal says: Hikr Backup: my personal thought
Sent 19 November 2014, 12h51
Could be many reasons to wish to backup your own reports on Hikr. You can keep in somewhere in your PC the documents, the pictures and other data you used to build your report; so you can think you are able to rebuild the report in case of Hikr servers failure, virus infection or other damages.
This is true but is not enough; what you lost is the Hikr page structure and design, the comments and the way to move through the pictures.
Some year ago I surfed the Web looking for some program helping to backup my reports in a better way. I found that WinHTTrack was the best program for this purpose; I saw that is a horse difficult to ride because you have to set many parameters to say it how to do the work but my son Marcello helped me and we gathered a satisfying parameters collection. After the initial effort you can always use those settings every time you backup a new report. Referring to the “Known issues” listed above WinTTrack solves all the points except the Minimap (Route) that is not shown if Hikr site is not reachable. But you can “save” your Route if it is listed under the Geodata and use it with other programs. You can use perfectly your backup without Internet connection and without Hikr server.
All the comments are kept, no matter if they are on the main page or under a picture.
All the picture caption you wrote are kept as well as the PDF documents if they are present.
The labels are in English even if on the original report you set another language; i.e. if you read “Data della gita:” on the original report you will read “Date of the hike:” on the backed up one but this not seems to be a problem.

The WinTTrack program is downloadable free of charge from the site:
http://www.httrack.com/page/2/en/index.html where you can choose the right product for you.

Below I list the parameters collection I use for my Hikr backup to give you a way ready to use (please note you have to exactly write the following parameters for the best result):

Limits:
Mirroring Depth: 3
Max Transfer Rate: 100000 B/s

Scan Rules:
Filters:
+*.png +*.gif +*.jpg +*.css +*.js -ad.doubleclick.net/* -mime:application/foobar
-*
-www.hikr.org/gallery/
-www.hikr.org/gallery/*
+www.hikr.org/gallery/photo*
+f.hikr.org/files/*.gpx
+f.hikr.org/files/*.kml
+f.hikr.org/files/*[0-9]s.jpg
+f.hikr.org/files/*[0-9].jpg
+f.hikr.org/files/*[0-9]l.jpg
+f.hikr.org/files/*[0-9].mov
+f.hikr.org/files/*[0-9].pdf
+s.hikr.org/*
-www.hikr.org/gallery/portal.html
-www.hikr.org/gallery/?mode=*
-www.hikr.org/gallery/tags/*
-translate.google.com/*
-f.hikr.org/files/
-www.hikr.org/tour/*
-www.hikr.org/gallery/?*
-*.flagcounter.com/*
-*.youtube.com/*


Italiano
Possono esserci molte ragioni per desiderare di fare un backup personale dei propri rapporti su Hikr. Voi potete conservare da qualche parte del vs. PC i documenti, le foto e gli altri dati che avete usato per costruire il vs. rapporto; in questo modo potreste pensare di essere in grado di ricostruire il report in caso di guasti ai server di Hikr, alla penetrazione di qualche virus o di qualche altro danno. Questo è vero ma non è abbastanza; ciò che perdereste è la struttura ed il design della pagina di Hikr, i commenti e il meccanismo per passare da una foto all’altra.
Qualche anno fa ho girato un po’ sul Web cercando un programma che potesse fare meglio il backup dei miei report. Ho trovato che WinHTTrack era quanto di meglio per questo scopo; ho visto però che non era facile cavalcare questo cavallo perchè dovevi caratterizzare molti parametri per dire al programma come fare il lavoro ma con l’aiuto di mio figlio Marcello abbiamo raccolto la giusta collezione di parametri. Dopo lo sforzo iniziale questi parametri vengono usati allo stesso modo ogni volta che si fa il backup di un nuovo rapporto.
Riferendomi ai “Problemi conosciuti” elencati sopra WinHTTrack li risolve tutti eccetto la Minimap che non viene mostrata se manca la connessione al sito di Hikr. Ma potete salvare la vs. Traccia se ne avete elencata una sotto Geodata e usarla con altri programmi. Per tutto il resto potete usare i vs. backup perfettamente offline, senza connessione internet nè connessione ad Hikr.
Vengono mantenuti tutti i commenti sia sulla pagina pricipale che quelli posti sotto la singola foto. Vengono inoltre mantenute le didascalie che avete scritto per le foto come pure eventuali documenti PDF. Le etichette del report salvato sono in Inglese anche se quando lo avete scritto erano in un’altra lingua; per intenderci se sul report originale leggevate: “Data della gita:” sul report salvato leggerete “Date of the hike:” ma questo non è un un problema.

Il programma WinHTTrack è scaricabile gratuitamente qui:
http://www.httrack.com/page/2/en/index.html dove troverete il prodotto che meglio si confà a voi.

Qui sotto elenco il set di parametri che uso per fare Hikr backup in modo da facilitarvi il compito (mi raccomando di scrivere i parametri esattamente come sono elencati).

Limits:
Mirroring Depth: 3
Max Transfer Rate: 100000 B/s

Scan Rules:
Filters:
+*.png +*.gif +*.jpg +*.css +*.js -ad.doubleclick.net/* -mime:application/foobar
-*
-www.hikr.org/gallery/
-www.hikr.org/gallery/*
+www.hikr.org/gallery/photo*
+f.hikr.org/files/*.gpx
+f.hikr.org/files/*.kml
+f.hikr.org/files/*[0-9]s.jpg
+f.hikr.org/files/*[0-9].jpg
+f.hikr.org/files/*[0-9]l.jpg
+f.hikr.org/files/*[0-9].mov
+f.hikr.org/files/*[0-9].pdf
+s.hikr.org/*
-www.hikr.org/gallery/portal.html
-www.hikr.org/gallery/?mode=*
-www.hikr.org/gallery/tags/*
-translate.google.com/*
-f.hikr.org/files/
-www.hikr.org/tour/*
-www.hikr.org/gallery/?*
-*.flagcounter.com/*
-*.youtube.com/*


83_Stefan Pro says: RE:Hikr Backup: my personal thought
Sent 19 November 2014, 16h47
That sounds very interesting! I think, I'll give it a try next time when snow makes a trip unpossible... best regards and thanks a million!

gbal says: RE:Hikr Backup: my personal thought
Sent 19 November 2014, 16h55
I'm happy to give you an interesting work to do next winter Stefan.
Bye
Giulio

giorgio59m (Girovagando) says: RE:Hikr Backup: my personal thought
Sent 19 November 2014, 17h20
Well done, Giulio !
;)

gbal says: RE:Hikr Backup: my personal thought
Sent 19 November 2014, 17h26
Thanks Giorgio

Deleted comment

gbal says: RE:
Sent 19 November 2014, 18h47
Thank you very much wegener.
You don't need to compile a text file with the list of your reports.
WinHTTrack already gives you a file index.html that links you to the backed up reports stored into your PC.
Please see here: http://www.hikr.org/post88269.html

Deleted comment

gbal says: RE:
Sent 19 November 2014, 19h04
No. I backup a single report at a time.
It's useful to wait a couple a weeks from the publishing waiting for the comments; then I backup that report, only.
The backup operation is not so fast then I think to do an user backup would require a very long time.

carpintero Pro says: RE:Hikr Backup: my personal thought
Sent 19 November 2014, 19h49
Hi gbal

Thanks for your comment and sharing the settings of WinHTTrack. I d'like to add some comments... :)

> Referring to the “Known issues” listed above WinTTrack solves all the points except the Minimap (Route) that is not shown if Hikr site is not reachable. But you can “save” your Route if it is listed under the Geodata and use it with other programs.

Just to clarify... the first point of the "Known issues" is refering to routes attached to waypoints - not the minimap. You'll find an example in the report *Chli Chärpf (2700m) – Hitzetour durchs Glarnerland where a route is attached to the waypoint  Chli Kärpf.

> The labels are in English even if on the original report you set another language; i.e. if you read “Data della gita:” on the original report you will read “Date of the hike:” on the backed up one but this not seems to be a problem.

You can specify the requested language on the "Browser ID" tab (in the same dialog as "Scan Rules"). Edit the value in the "Language" field - default is "en,*" (without quotes). Some of the possible values are "de", "en", "it", "fr", "es"... (of course, without quotes too).

I may mention here, that the hikr.org-backup script is supporting different languages too (the same way).

Best regards
Simon

gbal says: RE:Hikr Backup: my personal thought
Sent 20 November 2014, 10h15
Hi Simon,
thank you for first opening this subject; you explained your method then I added mine. I hope that the Hikrs now think about this important thing and decide how to behave. Thank you for the last suggestions and clarifications too.
Have a nice day
Giulio
Giulio

Sent 19 November 2014, 16h42
Hallo carpintero, vielen Dank, dass du dir diese Mühe gemacht hast und dein Ergbnis mit allen teilst. Das spricht für dich. Beste Grüße!

Bergpoetin says: Nice
Sent 20 November 2014, 11h23
But what about MacOS?

carpintero Pro says: RE:Nice
Sent 20 November 2014, 13h53
As far as i know, all the dependencies (ruby+gems, wkhtmltopdf) are available for OSX. It is possible, that the script works for Mac too (in fact, i expect it to work), but i didn't test it until now. Unfortuntely i don't have the possibility to test it by myself, because i don't have a Mac and it is not easy to virtualize OSX.

SCM says: RE:Nice
Sent 20 November 2014, 17h10
I tested it with OS X Yosemite and it worked fine. I used homebrew to install all the dependencies. The only problem I encountered was the installation of nokogiri which seems to be a common problem. This post helped me a lot: http://stackoverflow.com/questions/5528839/why-does-installing-nokogiri-on-mac-os-fail-with-libiconv-is-missing?rq=1

carpintero Pro says: RE:Nice
Sent 20 November 2014, 21h39
Wow, thank you very much for testing this!

SCM says: RE:Nice
Sent 21 November 2014, 11h17
No problem. So you can add the OSX installation to your readme file. If you need help, I can provide you with the exact terminal commands i used.

MatthiasG says: fail
Sent 1 December 2014, 15h58
I have tried to install this on a windows machine without direct internet connection and failed to install nokogiri. First it fails to download zlib. Using the parameter --use-system-libraries it then tells me I am missing libxml2, but when I try to install this I get the missing zlib again, even though it seems to recognize the --use-system-libraries parameter...

Giving up now. I am a software developer and this took me half an hour so far. I think non-IT-people can save themselves the trouble to even try installing this without open internet connection. It would really be nice to have an offline installer for this.

carpintero Pro says: RE:fail
Sent 3 December 2014, 20h33
Hi Matthias

nokogiri seems to be a hard to install (on OS X as well). If online, usually it's no problem installing it on Windows - even no build environment is required, because prebuilt binaries are included in the package.

Unfortunately, this depends also on Ruby version you have installed on your machine. At least for now, i recommend using Ruby 2.0.0-p598 (or the 64-bit variant), downloadable from http://rubyinstaller.org/.

Maybe i will replace nokogiri by an alternative in the future.

Thanks for feedback anyway,
Simon

MatthiasG says: RE:fail
Sent 4 December 2014, 15h18
Thanks for the info! I'll give it another try then.

carpintero Pro says: RE:fail
Sent 4 December 2014, 21h13
It is a bit cumbersome with this Ruby stuff... with the Ruby version above, it will give you most likely a SSL certificate error if you're trying to install Ruby Gems. Changing the Gems sources to the non-HTTPS worked for me (of course at the cost of the security risk):

> gem sources -r https://rubygems.org/
> gem sources -a http://rubygems.org/

carpintero Pro says: RE:fail
Sent 5 December 2014, 11h06
I learn everyday something new... :)

Today it's again about the version of Ruby on Windows.
I strongly recommend to use Ruby 1.9.3, at least on Windows. This version is also recommended by the site http://rubyinstaller.org/ and seems to work without troubles (no problems with nokogiri, no troubles with SSL certificate error).

I will update the README.md as soon as possible.

carpintero Pro says: Neues Merkmal / New Feature
Sent 1 December 2014, 20h21
NEU: Erstellt eine Text- und eine CSV Datei des Journals
NEW: Creates a text- and a CSV file of the journal


Post a comment»