Redesigning Interaction [Intangible Interaction]
The Intangible Interaction of Popular Voice Assistants (Siri & Alexa)
Research and writing done in collaboration with Bruce Arthur whose blog you can find linked here.
Siri works through a combination of AI and language processing. Spoken phrases are converted into data files, and sent to apple servers where they can be decoded in language processing and responded to in kind. Settings can be adjusted to always be listening so that the interaction can be triggered remotely through the phrase “hey Siri”. Similarly, Alexa makes use of Automatic Speech Recognition which converts speech audio into text files that are then processed using NLP (natural language processing) to understand a user request before responding with a process or simulated speech (Source: nymag).
Siri works through a combination of AI and language processing. Spoken phrases are converted into data files, and sent to apple servers where they can be decoded in language processing and responded to in kind. Settings can be adjusted to always be listening so that the interaction can be triggered remotely through the phrase “hey Siri”. Similarly, Alexa makes use of Automatic Speech Recognition which converts speech audio into text files that are then processed using NLP (natural language processing) to understand a user request before responding with a process or simulated speech (Source: Amazon).
Siri is an intangible interaction that uses voice commands to activate certain functions of and iPhone and other apple products. The interaction can be initiated by uttering the phrase “hey Siri” to a device that has the proper settings enabled. From here, a user can control apps within the phone or the phone itself. For example saying, “hey siri, skip to the next song” will prompt the listening device to skip a track in many apple supported streaming services like apple music, SoundCloud, and Spotify. The interaction will also alert you that it has received a command through lights and sound, even sometimes going as far as to speak back to you depending on the request or its response. Similarly, Alexa makes use of voice recognition technologies to allow users control of software, and in some cases hardware, applications after an initial utterance like “Alexa” is made. This initial utterance is not subject to full-throttle speech recognition itself but rather a local program which compares audio waves, prompting speech recognition once the set phrase is spoken and recognized (Souce: nymag).
Bruce Opinion: Personally I have no problem with the system of siri, but I do find the idea of an always-on listener a little problematic. Especially in a generation of targeted advertising and general internet security concerns, I think its giving tech companies like apple a little too many privileges to take the passive conversation and use that to improve their AI systems. The interactions, while useful, I don’t believe are worth my privacy.
Sim Opinion: I’ve watched the use of systems like Alexa and Siri provide tremendous support to folks in my family with limited mobility and many responsibilities and I think that the system of voice control has great potential to bring a little bit of aid to many folks in similar situations. However, as discussed by Bruce, the question of privacy and security of speech documents is quite concerning, especially when this technology is developed by large corporations with known prior violations of privacy and data. In the households in my family that include children who make use of Alexa, I am glad that they are able to make use of the tools for self-learning and to connect with family and friends yet I worry greatly about their future in terms of their data and their right to have control of that.
Additionally, the unreliability of the voice recognition technology with users of various dialects makes me question why the eagerness to constantly push and expand these products into the market. Detailed in Ruha Benjamin’s Race After Technology, a former Apple employee working on improving voice assistant technology for various dialects of English questioned the exclusion of African American english, to which a supervisor responded “Well, Apple products are for the premium market.” (Chapter 4: Why Now?, Page 32). In addition to this documented willful shortcoming of Apple to develop for Black Americans who speak AAVE, I’ve witnessed secondhand the extreme faultiness of Alexa when used by my grandmother who speaks English as a second language from a Spanish speaking background. Releasing widespread access to a technology that has the power to control so many different processes like sending messages or making purchases, knowing that it will be faulty for large percentages of the u.s. (and thus market) population is quite irresponsible in my opinion with the potential to do more harm than help for some communities.