Joe last caught up with the Bookshare team in Las Vegas with the announcement of the service hitting the one million book mark. As impressive as that feat is, Benetech Product Manager Nicholas Bowen provided Joe with an initial look at how Bookshare is working to make math easier to read. Listen in, or read below, how cloud services are being used to scale in order to allow the new "Math Detective" the ability to use OCR for a MathML injection into the appropriate area within a textbook. To learn more about this new tool, sign up for the newsletter at the official Bookshare site as new information will be released through that outlet.
ATIA 2020 coverage is Brought to you by AFB AccessWorld.
For the latest news and accessibility information on mainstream and access technology, Apple, Google, Microsoft, and Amazon offerings, access technology book reviews, and mobile apps, and how they can enhance entertainment, education and employment, log on to AccessWorld, the American Foundation for the Blind's free, monthly, online technology magazine. Visit www.afb.org/aw.
Transcript
We strive to provide an accurate transcription, though errors may occur.
Hide transcriptTranscribed by Grecia Ramirez
From beautiful, and sunny? -- cloudy Orlando, it’s blindbargains.com coverage of ATIA 2020, brought to you by AFB AccessWorld.
For the latest news and accessibility information on mainstream and access technology; Apple, Google, Microsoft, and Amazon offerings; access technology; book reviews; and mobile apps and how they can enhance entertainment, education, and employment, log onto AccessWorld, the American Foundation for the Blind's free monthly online technology magazine. www.AFB.org/AW.
Now, here’s Joe Steincamp.
JOE STEINCAMP: Back here from the show floor at ATIA, Orlando 2020. And I’m here with Nicholas over at Bookshare, and we have something really interesting to discuss, don’t we? There’s been some changes going on at Bookshare. Let’s talk about – oh, I don’t know, something Joe is not good at, which is usually orientation or math, so our listeners know it could be either.
NICHOLAS BOWEN: What do you mean you’re not good at math?
JS: Ah, I was never great at math. 780 verbal, 240 for math, and 200 for just signing your name.
NB: Well, at least you spelled your name right. That’s the important part.
JS: I know. That’s how I got the verbal.
NB: So math at Bookshare has been a longstanding problem. And the problem for us isn’t so much needing to know how to make the math accessible, it’s needing to know how to do it scalable. Because Bookshare has tens of thousands of math books, each math book has tens of thousands of images in it. And to try to get all that remediated just was far too costly. We just could not do it. And we – there are tools out there that are math OCR engines that we could ship all the images off to, but if we did that, again, it would be too costly still, because even that – half the images aren’t actually math equations.
So this is where Math Detectives comes in. We’re announcing this here at ATIA. We're going to hopefully launch this with Bookshare within the next couple of months. Math Detective is a machine-learning classifier where it classifies all the images you send to it to determine which ones have math and which ones do not. The ones that have math, it automatically sends to a math OCR engine and pulls back the MathML and LaTeX. So we’re going to take the MathML and inject that back into Bookshare books, replacing the equation, instantly making the entire math book MathML accessible.
JS: That is amazing. Oh wow.
NB: And the whole point of Math Detective is cost savings. So not only are we doing that, we’re caching everything, because so many math books use the same equation over and over and over again. The image classifier that we’re using costs a fraction of what the math OCR engine does, so we’re going to cache every single equation that we get in. And the next time we see that equation, instead of sending it out to be OCR’d again, we’re just going to give them what we got the last time because that’s going to save us immensely as we go.
JS: And then people can report it if it’s off, and you can go back and correct it and add –
NB: Well, absolutely.
JS: The crowd source at work.
NB: Yup.
JS: Wow. That’s amazing. Okay.
NB: Now, the other fun part is, talking about the future, what this actually means. So at a high level, what we’ve done is we’ve created a machine-learning tool that sorts images into one of two piles. And one pile, we know we can make accessible; the other pile, we can’t. But why can’t we? There are so many other tools that could be used with images to turn different kinds of images accessible. So Math Detective is being announced today, but in a very near future, it’s going to be Image Detective, because all we need to do now is classify tables and send the table out to be turned into a fully HTML table, and now we’ve made all tables fully accessible. Then we’ve got basic graphs, line graphs. What if we created a description engine that said, "This is a line graph facing up. This is a parabola facing up.” We’re doing that right now. What if we could do basic shapes? “This is a circle; this is a square.” We could do that too. That’s real easy.
There are already image identification services out there like Microsoft and Google Vision where we could send basic images to and have them generate for us basic alt text. Why not? At a high level, we’ve created the infrastructure for that. Math Detective builds on Amazon serverless infrastructure, meaning it infinitely scales to demand, there’s no limit. And it’s an open API. We’re looking for partners here on the floor. Anybody who sees a need where they’re going to get a bunch of images in, and they want to make them accessible.
JS: Wow.
NB: Anybody can integrate with us.
JS: That is amazing. And like you said, to scale -- and you have a good idea, with telemetrics, where your user base is –
NB: Uh-huh.
JS: -- what your load is going to be?
NB: Yup.
JS: Wow. Okay.
NB: So we’re very excited, obviously.
JS: Yeah. And I’m with you. That is a really amazing feat. And so the roll-out plan -- and changes coming as you go –
NB: Yup.
JS: -- as new text comes in. And then there’s already, I imagine, a significant portion of the existing library that you’ve already tested on and tested against.
NB: We are working on the existing library as we speak, on our testing entitles –
JS: Yeah.
NB: We're already replacing images with MathML and generating fully accessible EPUBs. We’re going to start with EPUBs.
JS: Okay.
NB: Our goal is once we get that in a stable state, which I’m hoping to find out within the next month or so –
JS: Good.
NB: -- we’re going to be working with our partners that Read our EPUB – because having a MathML embedded EPUB is useless if there’s nothing that reads it.
JS: Sure.
NB: So we’re going to be working with our partners to make sure that it works with their platforms as well.
JS: Yeah.
NB: And then we’ll launch. We will probably end up launching two versions of EPUB, the Bookshare standard EPUB and then the math accessibility EPUB. That way, our users that are on platforms that don’t get read MathML don’t end up in a worse situation than they are today, but then the users that are on a math-ready device can read a fully MathML accessible book. And then once that’s done, then we move on to the next part and do everything else that we think this tool can do.
JS: Can users already go see some examples of that in action or –
NB: That –
JS: -- is there sample page that people can go look at?
NB: Not yet. Stay tuned to the Benetech newsletter. We’re going to announce the full roll-out of the Math Detective website, and on the website will be some examples. But that’ll -- that’ll be out in another couple of months once we get a few more of these kinks worked out.
JS: That is amazing and a reason to go get that newsletter right there.
NB: Oh yeah.
JS: Anything we didn’t touch on, we need to circle back on, or are we good?
NB: I think we’re good. I –
JS: I can’t imagine –
NB: We’re just very excited.
JS: I can’t imagine what else you could add to that. My mouth has been on the floor half this time.
Nicholas, that’s amazing. Thank you for taking some time to explain it to us. That’s why we came by.
NB: Yeah. Of course.
JS: I appreciate your time, and thank you so much for the information.
NB: And thank you. Thank you.
JS: You bet.
For Blind Bargains, this is Joe Steincamp on the floor here at ATIA, just mouth aghast. It’s – that’s really amazing. Stay tuned. We’ve got more coverage, or go backwards in the feed. You’ll find more interviews from ATIA2020.
For more exclusive audio coverage, visit blindbargains.com or download the Blind Bargains app for your iOS or Android device.
Blind Bargains audio coverage is presented by the A T Guys, online at atguys.com.
This has been another Blind Bargains audio podcast. Visit blindbargains.com for the latest deals, news, and exclusive content. This podcast may not be retransmitted, sold, or reproduced without the express written permission of A T Guys.
Copyright 2020.
Listen to the File
File size: 8.2MB
Length: 07:39
- Download the file
- Listen using our Enhanced Accessible Web Player
- Download our Blind Bargains Apps to listen from your phone or tablet
Check out our audio index for more exclusive content
Blind Bargains Audio RSS Feed
This content is the property of Blind Bargains and may not be redistributed without permission. If you wish to link to this content, please do not link to the audio files directly.
Category: ShowsNo one has commented on this post.
You must be logged in to post comments.
Or Forgot username or password?
Register for free
Joe Steinkamp is no stranger to the world of technology, having been a user of video magnification and blindness related electronic devices since 1979. Joe has worked in radio, retail management and Vocational Rehabilitation for blind and low vision individuals in Texas. He has been writing about the A.T. Industry for 15 years and podcasting about it for almost a decade.
