Haitian Earthquake 12 January 2010

Here wizards, magicians, sorcerers and everybody can rest a bit and talk about anything they like.

Just remember to respect the rules.

Haitian Earthquake 12 January 2010

Postby Weatherlawyer » Jul 13th, '21, 09:54

I am trying to download a run of weather charts from:
https://digital.nmla.metoffice.gov.uk/S ... 3aec7cfb0/

To see the surface pressures from the first of the month to the last and eventuall all the folder from the previous year to thhe end of2010. I have no idea if it can be done nor how to do so Can anyone explain to me how, if it is possible?
Thanks in advance.
Weatherlawyer
 
Posts: 90
Joined: Mar 21st, '14, 00:04

Re: Haitian Earthquake 12 January 2010

Postby doktor5000 » Jul 13th, '21, 17:22

Sure it's possible. Question is if you want to do it manually or in a scripted way. For the manual way, feel free to do it as you wish.
For the scripted way, you would need to grab all the links from e.g. the folder for 2010 - https://digital.nmla.metoffice.gov.uk/s ... c86ba759d/

Here's an example:
Code: Select all
URL=https://digital.nmla.metoffice.gov.uk/so_08948fb4-2453-4576-a021-1b2c86ba759d/
wget -q $URL -O - | grep -o -E 'href="([^"#]+)"' | grep /so_ | cut -d'"' -f2


There are also other approaches e.g. via
Code: Select all
lynx -dump -hiddenlinks=listonly $URL


Then you would need to go through each of those links, and the sub-pages and fetch every link to a picture below those, and download them.

You could also try to crawl/mirror a subset of that website, but that requires a bit of understanding how the website works to limit what you download and also how to view it later on.
See e.g. https://www.guyrutenberg.com/2014/05/02 ... sing-wget/ for an example.
Cauldron is not for the faint of heart!
Caution: Hot, bubbling magic inside. May explode or cook your kittens!
----
Disclaimer: Beware of allergic reactions in answer to unconstructive complaint-type posts
User avatar
doktor5000
 
Posts: 17603
Joined: Jun 4th, '11, 10:10
Location: Leipzig, Germany

Re: Haitian Earthquake 12 January 2010

Postby Weatherlawyer » Jul 13th, '21, 22:07

I tried the one by one method from another man's site:
http://www.woksat.info/etcsaasxx/indexsaasxx.html

I found it nearly impossible to see what I had copied until It refused to double-up. No with it problem overwriting but he occasional wrong url grapic ending .JPG or whatever, is going to make it difficult to assmeble a series.

Wouldn't you think a resposnible body asvertising tax paid services would make their archives easily accessible but they don't even offer the advice you have supplied
I thank you for this help it has save me an awfully redious proble.

When I ge the stuff postec I will bring back a link thanks.
Weatherlawyer
 
Posts: 90
Joined: Mar 21st, '14, 00:04

Re: Haitian Earthquake 12 January 2010

Postby doktor5000 » Jul 13th, '21, 22:29

Weatherlawyer wrote:Wouldn't you think a resposnible body asvertising tax paid services would make their archives easily accessible but they don't even offer the advice you have supplied


Did you actually ask them beforehand for what you wanted, or are you only assuming ?
Cauldron is not for the faint of heart!
Caution: Hot, bubbling magic inside. May explode or cook your kittens!
----
Disclaimer: Beware of allergic reactions in answer to unconstructive complaint-type posts
User avatar
doktor5000
 
Posts: 17603
Joined: Jun 4th, '11, 10:10
Location: Leipzig, Germany

Re: Haitian Earthquake 12 January 2010

Postby Weatherlawyer » Jul 13th, '21, 22:49

To be honest I misjudge people all the time. I am what the bible calls a curmudgeon But why the hell would people assume lamers like can even find their bloody pages, never mind see and underswtand how important they are. Who on earth would believe that Elite murderers are using weather control to steal children?

And why do you assume that Government agencies are not aiding cannibalism?
You must know by now, that the US elections were fraudulent. How many other countries have been robbed like that?

But you are right. I will have to ask but the very first time I approached such people it didn't go well. It never has so I beg you to overlook my deep suspicions about them.
Weatherlawyer
 
Posts: 90
Joined: Mar 21st, '14, 00:04

Re: Haitian Earthquake 12 January 2010

Postby Weatherlawyer » Jul 13th, '21, 22:58

doktor5000 wrote:
Here's an example:
Code: Select all
URL=https://digital.nmla.metoffice.gov.uk/so_08948fb4-2453-4576-a021-1b2c86ba759d/
wget -q $URL -O - | grep -o -E 'href="([^"#]+)"' | grep /so_ | cut -d'"' -f2




First link fail:
URL=https://digital.nmla.metoffice.gov.uk/so_08948fb4-2453-4576-a021-1b2c86ba759d/
wget -q $URL -O - | grep -o -E 'href="([^"#]+)"' | grep /so_ | cut -d'"' -f2


Try the second next
Weatherlawyer
 
Posts: 90
Joined: Mar 21st, '14, 00:04

Re: Haitian Earthquake 12 January 2010

Postby Weatherlawyer » Jul 13th, '21, 23:04

Weatherlawyer wrote:
doktor5000 wrote:URL=https://digital.nmla.metoffice.gov.uk/so_08948fb4-2453-4576-a021-1b2c86ba759d/
wget -q $URL -O - | grep -o -E 'href="([^"#]+)"' | grep /so_ | cut -d'"' -f2
t



lynx -dump -hiddenlinks=listonly $URL

command not found
I have no idea how helpful my replies arewhen replies to Konsol are not complete.
Weatherlawyer
 
Posts: 90
Joined: Mar 21st, '14, 00:04

Re: Haitian Earthquake 12 January 2010

Postby doktor5000 » Jul 13th, '21, 23:06

How does it fail, what's the actual error ?

For lynx, you would have to install that if you want to use it.
Cauldron is not for the faint of heart!
Caution: Hot, bubbling magic inside. May explode or cook your kittens!
----
Disclaimer: Beware of allergic reactions in answer to unconstructive complaint-type posts
User avatar
doktor5000
 
Posts: 17603
Joined: Jun 4th, '11, 10:10
Location: Leipzig, Germany


Re: Haitian Earthquake 12 January 2010

Postby Weatherlawyer » Jul 13th, '21, 23:15

doktor5000 wrote:
Weatherlawyer wrote:Wouldn't you think a resposnible body asvertising tax paid services would make their archives easily accessible but they don't even offer the advice you have supplied


Did you actually ask them beforehand for what you wanted, or are you only assuming ?


https://digital.nmla.metoffice.gov.uk/contact-us/
This is a joke designed as the usual circle jerk
Weatherlawyer
 
Posts: 90
Joined: Mar 21st, '14, 00:04

Re: Haitian Earthquake 12 January 2010

Postby doktor5000 » Jul 14th, '21, 17:45

Weatherlawyer wrote:https://digital.nmla.metoffice.gov.uk/contact-us/
This is a joke designed as the usual circle jerk

So you didn't ask them about your request, because those people should be able to read your mind and know what you want from them in advance, and they should present the data you want in a way that is convenient to you, and for free.
But you still choose to complain about tax paid services and stuff. Perfectly valid approach :roll:

For the output you posted, that is a list of the links of the folders containing the links to the images, one for each month. Seems you skipped the following part of the explanation in my reply:
doktor5000 wrote:Then you would need to go through each of those links, and the sub-pages and fetch every link to a picture below those, and download them.
Cauldron is not for the faint of heart!
Caution: Hot, bubbling magic inside. May explode or cook your kittens!
----
Disclaimer: Beware of allergic reactions in answer to unconstructive complaint-type posts
User avatar
doktor5000
 
Posts: 17603
Joined: Jun 4th, '11, 10:10
Location: Leipzig, Germany


Return to The Wizards Lair

Who is online

Users browsing this forum: No registered users and 1 guest

cron