Kiwa Media, the iPad & QBook
Saviour or destroyer? iPad and books
There are books that are basically just digitised flick-through books, but there are also ventures like Nick Cave’s Bunny Munro, which combines bookmarking with video of Nick reading his work. Stop the video and go back to the text, and it goes to the part of the text where you stopped Mr Cave’s delivery. Clever. (It’s $20.99 but you can get the free Lite version which gives you a good idea of what to expect.)
One of the iPhone book companies that has been particularly innovative is Kiwa Media, which happens to be based in Newton, Auckland, New Zealand.
Luke Tomes, VP Technology of Kiwa Media, recently talked to me about the company’s QBook and CUT developments.
QBook is an ebook for kids, but development began a year ago with the Apple’s iPad firmly in mind. This was based on rumours but also traces of code found in the iPhone SDK hinting at what was to come. (The QBooks look great on iPhone and iPod touch – the big advantages of the iPad is, simply, size.)
Kiwa's Hollywood software
Kiwa started up about ten years ago, developing Mac software called VoiceQ, a specialist tool used in the film industry. It’s for dubbing films into multiple languages and for ADR – dialogue replacement. This is when you get the actors back to re-voice their dialogue in moving footage, to give the sound designers better quality voice audio to edit with.
With VoiceQ, you enter in the script as a text file, and align the words to the original speech waveforms using the software. Then it actually plays the movie and scrolls the words across the bottom of the screen, for the actor to voice to, in sync with the actor on screen.
“We’ve been using it internally to do the dubbing for a lot of the imported programs on Maori Television. But it’s been sold into Hollywood Studios like Fox and Disney – we have customers all over the world. It’s Mac only and it integrates with Pro Tools.”
“You can add a new language, so when the dialogue scrolls across it’s in Italian or whatever. So then it can be revoiced in a new language.”
VoiceQ (which is patented) was used, for example, in Tom Cruise’s movie Valkyrie, and Luke went over to Europe to help with the dubbing as the movie was being made.
Once reason it’s Mac-only is that, says Luke, Mac OS X contains great localisations – switch the OS to Mandarin and the system terms and dialogue boxes are all in Mandarin – or Cherokee, or Armenian, etcetera.
After VoiceQ, Kiwa launched SingQ, which stretches the words horizontally so you know how long to hold a note. Videos that result can be rendered out to cell phones – anything that can run video – and it's a technology that may revitalise Karaoke.
iPhone apps
SingQ led Kiwa to look at iPhone apps and the music world. They realised that digital publishing (words, in other words) was about ten years behind the music industry. “We realised there was little beyond making a work into a PDF or a basic ebook. So we started to consider how we could apply some of our intellectual property to synchronised visual text and audio, which is where we specialise, and in multiple languages.
“Early QBooks had scrolling text going along the bottom and you could read it when it hit a line.”
First of all, moving text is distracting. Secondly, it dictates the pace at which you follow the story, which is not ideal. “And I suddenly thought ‘what if we ran our finger along the text, and as it touched, it said the word?’ And we’ve never seen that anywhere else. As soon as we programmed up a prototype, we realised it was brilliant.”
In use, kids start to associate the look of the words with what they hear. They can ‘read’ at their own pace and go back as often as they want. “You can touch any word and it says it, and if you touch and hold on a word, it spells it out to you. And that’s every word in the book. And that’s how we arrived at the QBook as an ebook for kids. ”
And it’s perfect for kids’ books because you can’t fit that many words on an iPod touch or iPhone screen anyway. I've looked at Milly Molly – it really works well.
This is a natural fit for kids’ books, but then Kiwa started working with multiple languages. “I’m having great fun with Spanish. I can start in English and then flip it into Spanish."
Not yet implemented in QBook, this is currently being worked on as a language learning tools.
"With two languages side-by-side, you [will be able to] read in English and when you run your finger along the English, the corresponding word is highlighted in Spanish.”
Kiwa has been working on adding Japanese, which has been “Incredibly hard. There are different character sets, many more sounds, and of course the Japanese characters weren’t supported on the iPhone.
“We haven’t just made one book of this – we’ve made a whole system that allows us to migrate content into this platform. And now we’re working with Penguin on this, plus Milly Molly and Huia Publishing, both independent New Zealand publishers.”
Kiwa takes their content, loads it into the CUT tool (Kiwa’s Content Universal Tool), which manages all the audio files and mapping, plus the back plate of each page. “We actually have people working here can make these books without reading a line of code, in a WYSIWYG editor. Then they hit ‘build’ and it fires up Xcode and it builds the app for the iPhone.”
NZ programmers and developers
Kiwa has been working with CSI academy at Auckland University. The Centre for Software Innovation supplies students as interns. “All of our programmers are people who have gone through CSI and worked here as students, then we’ve hired them when they finished their degrees in Computer Science.”
CSI is not a Mac campus. “They don’t really teach Mac programming anywhere [in New Zealand], which seems incredible, but the job options in the main market are limited. That said, I know Mac programmers who start work at $1500 a day. If you’re good, it’s quite a lucrative business. We specifically ask for programmers with an interest in Apple technology, but where we can’t find them, we convert them.
“That said, we’re looking at taking the system to Windows 7. Because it’s touch-enabled. So we want to bring QBook to Windows 7. It’s going to be a bit of work, but basically it’s just a matter of adapting. And we always made the system with that in mind. We have been working in Objective C for Mac and we can adapt to C++ for Windows. We separated out the whole media framework from the outset, and the CUT tool will actually convert all the pictures to the right size and format for the platform desired.
“We’re also researching Symbian and Android, and we also built all out tools with that in mind. We want any one QBook to be able to exist on multiple platforms and to be consistent across those platforms as far as possible within different systems’ limitations. Touchscreen, of course, is required.
“That means that our customers, meaning publishers, are not having to use a different company to develop for each platform and constantly having to reinvent everything each time. The image formatting and sound formatting is a lot of work, so we only do that once, then we make this Master Media Framework, and when we ‘build’ the project it reference that and creates the assets required.”
iPad
"So we’re actually all ready for iPad. We’re testing on the iPad simulator. The day they announced it, we downloaded it. It was relatively painless to get our system running on it. It did introduce some new technologies. One of the things that wasn’t available for our previous iPhone development is CoreText, like other Core technologies introduced by Apple.”
(That’s CoreAudio, CoreVideo, CoreGraphics, CoreData etc, all things that streamlined the programming process, says Luke.)
macnz: Will this CoreText become available on Macs?
“Yes. And it’ll be in the next iPhone OS too. We had to actually write a whole method to take the individual glyph characters for each font in CUT to render, for example, a Japanese character. Because there wasn’t anything in iPhone to let you render that unless you mean user interface elements (in menus, etc). Unfortunately, that was all made redundant when CoreText came out, but we didn’t know that was coming. But that’s what happens.”
Kiwa Media has five full-time developers and five full-time production editors doing Photoshop work, audio recording and syncing, and I have two managers constructing the books. And Roger Shakes doing business development, working with publishers and customers. It was a three or four-man band for a while but the last two years has seen a lot of growth.
macnz: Do you think there’s a future for adult books for iPad etc?
“Yes I do. Absolutely. The epub format is great for reading novels and stuff like that, but even reading manuals or textbooks on the iPad is going to be brilliant. I definitely think it’s going to be big in the educational arena.
“The possibilities for people with disabilities, too – there’s lots of other things it can help with. I think the iPad has been designed from the user’s point of view backwards, rather than tacking on a user interface on a system designed by a computer engineer. A computer engineer will lay things out a certain way, according to logic, and then an interface is tacked on top. With Apple’s iPhone touch interface, users intuitively get it.”
Since most computer users these days don’t very often need to input, the iPad will be a dream device for many.
Open Systems and that
“The whole platform gets panned on SlashDot for not being ‘open’. They say. But that is complete rubbish. For $160 I can buy a development kit and I can make apps and put them on my iPhone, and I can install them on any iPhone in the office. If I want to sell it, I have to go through it the App Store approval process. But fair enough – they invented it. They get their cut.
On the recent App Store deletions of ‘nudie apps’, Luke says “That’s quite good. You can’t be selling kids’ apps and porn in the same store. So good on them. And you’ve got Safari and Google – two clicks and you’re away, anyway.
“I’d better get some orders in now for the iPad. I actually have someone standing by in the States to get some for us if they don’t go on sale here.
“We’re actually going to the London Book Fair and take the iPad there and show QBook.”
Getting content into QBook
“You bring the content to us and we do it for you. And we actually manage the iTunes account for them, too. Basically because the companies don’t know how to do it. But I imagine that in the future people will want to manage that for themselves. For now we’re getting a return on our quite significant investment. “This system means we can be knocking out ten or 20 books a week. There are a lot of people out there making apps, but they’re one-off apps. The great thing is that when we update the QBook framework or the engine, like make an improvement in the interface or add a module, we can just hit the build button again and roll out all those books again as updates.
“We did look at a model where we had a player, into which people loaded individual books …” but Kiwa decided to stick with the self-contained apps so people can manages them from the iTunes interface.
macnz: If I gave you a book, how long would it take to output a QBook?
“We can do it from a PDF, but we prefer Adobe InDesign files. We tell people two weeks, but we can do it in a week.
“We need to look at the story, find a good voice for it, book them, get them in to read it – but then it’s just a matter of making a back plate and chucking it in.
“I always wanted well-known voices to read them, but it comes down to budget.”
macnz: I can imagine a publisher paying for, say, Emily Perkins to come in and read her own work.
“Sure. Why not? That’d be great. Like Spike Milligan and Badjelly The Witch.”
Yeah! (Try out the free Sebastian's Tail Lite.)
