Download All Messages From HashBlue #Blue on O2

The new service from o2 for sending and storing messages online is cool but at the moment there is no easy way to grad all the messages. So I have created the following script to do just that. You will need to log into your HashBlue account and then go here:



When there click on the start link and you will see

"To perform this request with curl, use:

curl -i -X GET "https://api.hashblue.com/account" \
-H "Authorization: OAuth v9rty6aBqtY5Leru0ior6bhu8MAwf79m" \
-H "Accept: application/json"
"

The OAuth token is the value you will need to enter in the script below:

#!/bin/ksh

OAuth=v9rty6aBqtY5Leru0ior6bhu8MAwf79m
i=1
OUTPUT=/Users/cranie/hash-blue/messages.${i}.json
URL="https://api.hashblue.com/messages?since=2005-01-01T00:30Z"

curl -i -X GET "${URL}" -H "Authorization: OAuth ${OAuth}" -H "Accept: application/json" >> ${OUTPUT}
URL=`tail -1 ${OUTPUT} | sed -e 's/.*next_page_uri\"\:\"//g' -e 's/\"..*//g'`
i=$(( ${i} + 1 ))
OUTPUT=/Users/cranie/hash-blue/messages.${i}.json

while [ ${URL} != "" ]
do
curl -i -X GET "${URL}" -H "Authorization: OAuth ${OAuth}" -H "Accept: application/json" >> ${OUTPUT}
URL=`tail -1 ${OUTPUT} | sed -e 's/.*next_page_uri\"\:\"//g' -e 's/\"..*//g'`
i=$(( ${i} + 1 ))
OUTPUT=/Users/cranie/hash-blue/messages.${i}.json
done

Then run. When done you will have your messages in json format in like so:

cranie:hash-blue cranie$ ls -l
total 1744
-rwxr-xr-x  1 cranie  staff    723 23 Aug 21:37 GetMessages.sh
-rw-r--r--  1 cranie  staff  23744 23 Aug 21:38 messages.1.json
-rw-r--r--  1 cranie  staff  23983 23 Aug 21:40 messages.10.json
-rw-r--r--  1 cranie  staff  24515 23 Aug 21:40 messages.11.json
-rw-r--r--  1 cranie  staff  24243 23 Aug 21:41 messages.12.json
-rw-r--r--  1 cranie  staff  24413 23 Aug 21:41 messages.13.json
-rw-r--r--  1 cranie  staff  24063 23 Aug 21:41 messages.14.json
-rw-r--r--  1 cranie  staff  24141 23 Aug 21:41 messages.15.json
-rw-r--r--  1 cranie  staff  25327 23 Aug 21:42 messages.16.json

iPhone Stanza Recovery

With a recent update of the iphone OS, a jailbreak here and there saw my Stanza library lost. Most of the books were sourced from online catalogs, some of which could be easily recovered, but this was a pain. So I set out with the aim of recovering and backing up my library for future use. Also some of the solutions I found do not name the books properly, so when importing to Calibre it messes up. I have more or less got around this.

The first part was easy, Mac, iTunes and iPhone backup. Check, this will also work if you want to save your iPhone library to your PC. Anyway that's enough rambling, lets get on with the good stuff.

1. Mac - all the tools mentioned are available for Windows, I love command line, so you are on your own (with Cygwin) if you choose that route.
2. iTunes and a backup / iPhone Jailbroken. For Mac you can use "iPhone Backup Extractor" Its free, easy and does the job. I'll let you work it out, likewise use Diskaid to get to your Stanza library, again I'll leave the discovery up to you. If there is interest I will help out.
3. Calibre This is where we will import the books to. This is not a necessity (just makes it easier)
4. There are plenty of tutorials as well using: CalibredbExtract This again is a must if you plan on hosting / Dropboxing Catalogs.

Once you have your extracted Stanza library, you should see something like this:


drwxr-xr-x 393 user user 13362 20 Dec 16:38 0
drwxr-xr-x 393 user user 13362 20 Dec 16:40 1
drwxr-xr-x 392 user user 13328 20 Dec 16:41 2
drwxr-xr-x 389 user user 13226 20 Dec 16:43 3
drwxr-xr-x 394 user user 13396 20 Dec 16:44 4
drwxr-xr-x 393 user user 13362 20 Dec 16:45 5
drwxr-xr-x 390 user user 13260 20 Dec 16:46 6
drwxr-xr-x 397 user user 13498 20 Dec 16:48 7
drwxr-xr-x 385 user user 13090 20 Dec 16:49 8
drwxr-xr-x 387 user user 13158 20 Dec 16:50 9


These are where the books are stored. The epub, is inside each of these folders unpacked as .explode folder. What we need to do is pack it back up, the easy part thanks to: This Blog. The zip part I am grateful for, as well as figuring it out, saved me time. That blog tells you how to do it for each book, with a for loop, some creative shell scripting we can convert all the books back to epub. 1st part done, the next is to rename the and give it a more meaningful .epub name. A bit of searching the xml, sed and awk and we get there.

The script, I copied into each folder and ran from there, and then issued a mv *.epub ../books to put them into one folder. Again if people want me to alter / provide a more rounded solution e-mail me.

for file in $( ls -1 | grep explode )
do
    filer=$( echo ${file} | sed 's/explode//g' )
    zip -Xr9D ${filer}epub mimetype ./${file}/*

    if [ -f ./${file}/OEBPS/content.opf ]
    then 
        TITLE=$(cat ./${file}/OEBPS/content.opf | egrep "dc:title" | sed 's/<\/.*>//g' | sed 's/.*<.*>//g' )
        CREATOR=$(cat ./${file}/OEBPS/content.opf | egrep "dc:creator" | sed 's/<\/.*>//g' | sed 's/.*<.*>//g' | awk -F, ' { print $2 " " $1 } ')
        mv ${filer}epub "${TITLE} - ${CREATOR}.epub"
    fi
done

Thats it, simple, easy, to the point and gets the job done. With a little searching, creative license you can turn the script into a recover, build and publish and make your own repository! That's for another day though.

Working on a few of the Cydgets...

Facebook and Twitter should now be working and no strange things happening!

24Cydget (my favourite) is being worked on and should be better in the next few min. Am woking on a few other things with regards to 24 (good times).

Had a few requests - will look into them, one I am interested in, and while Elements will provide it, sometimes just one Element :-) is useful.

Any issues with the repo should be sorted (new versions - better make process now) - repo refreshed so mismatch should be gone.

Make Cydia Repo Script

Not sure if this is useful or not, but if you find yourself following the tutorials and want a simple ne click script to create your Packages.bz2 and deb files feel free to use the following script. I have the following structure:

/repo/apps/

containing all the apps

/repo/uploads/deb 

Which has the packages and deb files.

cd apps
find . -name .DS_Store -exec rm {} \;
for app in $( ls -1 . )
do
    dpkg -b ${app}
    mv *.deb ../upload/deb
done

cd ../upload/
rm Packages.bz2
dpkg-scanpackages . /dev/null > Packages
bzip2 Packages

Script should be placed (in the above example) in:

/repo/

Cydia Repo for Cydget's

OK. First repo for iPhone, contains 2 Cydget's:

Twitter
Facebook

Basically just loads each respective page. I am investigating the issue with the Twitter not showing the mobile version.

Repo:

http://cranie.com/repo/

Simple!

Get cydia packages from a repository

Ok, not sure how useful this is, or how it will be taken by repo hosts, but, I find, especially when restoring, that having the packages on my computer is very useful!

Script, call it something like getcydia.sh, and run as follows:

getcydia.sh cydia.reponame.com foldertostoredebs

Script is here:

#--- START ---
REPO=${1}
FOLDER=${2}

mkdir ${FOLDER}
cd ${FOLDER}
wget -U "Telesphoreo APT-HTTP/1.0.98" ${REPO}/Packages.bz2
bunzip2 Packages.bz2
cat Packages | grep Filename | sed 's/Filename: //g' | awk -v REPO=${REPO} ' { print "wget -nc -U \"Telesphoreo APT-HTTP/1.0.98\" " REPO "/" $0 } ' | sh
#--- END ---

http://pastie.org/646073

Restore from iPhone backup

3GS jailbreak is now out. I have just updated my phone and done a restore using iTunes. Needless to say this leaves a lot of junk left around. So I decided to restore again and set up my phone as a new one. I synced all my music, apps, vids and, as expected, none of the application save data was available.

So to the restore script. If you made a backup using the script below (and done all) you can run the following script to restore your data).

Run as:

./restore.sh 20091002

Where the 20091002 is the date folder of your backup. This is the first cut of the script, so its just to get the data onto the phone, it will be integrated into the main script. You will need to alter the IP to that of your iPhone:

#--- START ---

USER=mobile
IP=192.168.2.3
export SSH="ssh -i ./iphone ${USER}@${IP}"
export SCP="scp -r -i ./iphone ${USER}@${IP}"
export SCPR="scp -r -i ./iphone "
export SCPS=" ${USER}@${IP}"
SOURCE=${1}

for file in $( ls -1 ${SOURCE} | grep \.app )
do
    APP=$( echo ${file} | sed 's/§/ /g' )
    RDIR=$( ${SSH} "cd /var/mobile/Applications/*/${APP}/../ 2> /dev/null ; pwd" )
    if [ ${RDIR} != "/private/var/mobile" ]
    then
        echo "Restoring ${file}"
        ${SCPR} ${SOURCE}/${file}/Documents ${SCPS}:${RDIR}
    fi
done

for file in $( ls -1 ${SOURCE}/Media )
do
    echo "Restoring ${file} "
    ${SCPR} ${SOURCE}/Media/${file}/ ${SCPS}:/var/mobile/Media/
done
#--- END ---

http://pastie.org/641700