The magic of touch: how deafblind people taught us to ‘see’ the world differently during COVID

As some­one who is severe­ly deaf and com­plete­ly blind, I felt overnight I had lost a third sense, my sense of touch. To make mat­ters worse, peo­ple around me fad­ed away – voic­es had become so qui­et that there was an eerie sound­less­ness all around. Noth­ing was mak­ing sense any more.

Issy McGrath has type 2 Ush­er syn­drome. Com­plete­ly blind and severe­ly deaf, she has a pas­sion for music and plays the flute. Using a com­bi­na­tion of touch, smell and keen imag­i­na­tion – her “inner eye” – Issy says she fre­quent­ly sens­es things that are beyond the grasp of sight: the “almost sol­id” nature of the win­ter air in the morn­ing, or the enchant­i­ng atmos­phere of a frozen land­scape.

For Issy and many oth­ers like her, the COVID pan­dem­ic had a dev­as­tat­ing effect on day-to-day life. “Two-metre social dis­tanc­ing felt like the world had turned its back on me,” she recalls. “It was too far for me to reach out and touch every­thing around me. Yet it’s main­ly through touch that I get a sense of what a per­son is like.”

A retired teacher liv­ing in Glas­gow, Scot­land, Issy speaks poignant­ly about her COVID strug­gles in an audio diary that was part of my research into the expe­ri­ences of deaf­blind peo­ple dur­ing the pan­dem­ic:

As I approach my gar­den gate, feel­ing around for the latch to open it, a thought occurs to me. There is a pan­dem­ic sweep­ing the world and maybe I will catch the virus from this wood­en fence. Maybe it’s on the latch I have just touched. I shake my hands to free myself from these thoughts. I make my way back to my house and wash my hands thor­ough­ly, try­ing to free my mind of these fear­ful imag­in­ings.

‘You can feel the energy of things’

As a film­mak­er, I am con­stant­ly ques­tion­ing how and what we see – and what we don’t see. This has led me to work close­ly with deaf­blind com­mu­ni­ties around the UK, to under­stand how their view of the world dif­fers from every­one else’s – in an ocu­lar­ce­ntric soci­ety that priv­i­leges vision over all oth­er sens­es.

Per­ceiv­ing through touch takes time. By method­i­cal­ly stroking dif­fer­ent sur­faces, deaf­blind peo­ple build up a men­tal image not only of a per­son or object, but their place in the sur­round­ing room or land­scape. Deaf­blind people’s hands and skin are, I think, unusu­al­ly sen­si­tive to dif­fer­ent lev­els of rigid­i­ty, to the feel­ing of dif­fer­ent tex­tures, and to slight dif­fer­ences in move­ment or tem­per­a­ture.

John Whit­field, a train­ing offi­cer at Deaf­blind Scot­land, has been severe­ly deaf since birth and now has only 5% of his vision left. He describes how much con­cen­tra­tion is required to under­stand the world around him and keep up with con­ver­sa­tions. “Some­times that is very, very tir­ing,” he admits.

Because you are so con­scious of the restric­tion on your hear­ing and vision, your brain has to com­pen­sate – and your body is hav­ing to com­pen­sate too by get­ting infor­ma­tion in what­ev­er way it can. My sense of smell is height­ened, for exam­ple. You are just des­per­ate to get as much infor­ma­tion from the envi­ron­ment as you pos­si­bly can, so you will use any method.

For Roger Wil­son-Hin­dr, who lives with his vision-impaired wife in a small vil­lage in the Mid­lands of Eng­land, touch­ing means more than just receiv­ing sen­so­ry input or hold­ing on to infor­ma­tion. He says every tac­tile inter­ac­tion is a chance to form a new rela­tion­ship, adding that “touch and phys­i­cal con­tact take on greater sig­nif­i­cance if your eyes and ears are bad­ly dam­aged like mine”.

Corneal scars and glau­co­ma suf­fered dur­ing child­hood lim­it what Roger can per­ceive – he is able to see colour but with lit­tle def­i­n­i­tion. Trees, one of his favourite things, appear as a gold­en or green mass.

But when gar­den­ing, he can still “feel” the sea­sons through the bend­abil­i­ty, tex­ture and direc­tion of the stems and branch­es. He says there is a “mag­ic” to touch – “you can feel the ener­gy of things” – and that it’s not always just about mak­ing up for a lack of vision. Deaf­blind people’s tac­tile world con­tains much joy.

Imag­ine, then, the impact for Roger and all oth­er blind and deaf­blind peo­ple when COVID trans­formed the mean­ing of touch and prox­im­i­ty to oth­ers – from a life-enrich­er to a poten­tial life threat. As Issy puts it:

Social dis­tanc­ing meant the world both passed me by and left me con­stant­ly con­flict­ed. Do I allow peo­ple into my space so that I can inter­act and make sense of the world, risk­ing catch­ing the virus? Or do I ask peo­ple to respect the two-metre social dis­tance rule, and allow a creep­ing sense of iso­la­tion to over­whelm my emo­tion­al well­be­ing?

The importance of touch

There are two com­mon mis­con­cep­tions about deaf­blind peo­ple: that they require con­tin­u­ous assis­tance and are not easy to com­mu­ni­cate with. Dur­ing our research, we heard how these per­cep­tions con­tribute to their exclu­sion from wider soci­ety and can have a dam­ag­ing effect on their con­fi­dence. This was all made worse by the pan­dem­ic, as Issy explains:

Hold­ing someone’s hand pro­vides me with so much infor­ma­tion – to feel the fab­ric of someone’s cloth­ing means I can get a real sense of their being. Sud­den­ly [with the onset of COVID], to be so far away from the scent of their per­fume or the tex­ture of their hair … it was all gone. Even with the relax­ing of social-dis­tanc­ing, the joy I had in reach­ing out to touch and link arms with oth­er peo­ple has become sub­dued and cau­tious, as I war­i­ly nav­i­gate my world through my sense of touch.

When we think about touch, we usu­al­ly think of hands and fin­ger­tips. But Roger high­lights that, for deaf­blind peo­ple, “touch uses all aspects of our body – from the top of our head to feel the sun­light, to our feet for feel­ing where we are on the street”. Indeed, all of our inter­vie­wees empha­sised the impor­tance of touch­ing with their feet – help­ing them to scan and per­ceive the envi­ron­ment while walk­ing, to recog­nise the char­ac­ter­is­tics of dif­fer­ent spaces and cre­ate a men­tal map.

As the first lock­down was eas­ing, Issy recalls being reduced to tears in the mid­dle of a street in her sud­den­ly unfa­mil­iar Glas­gow neigh­bour­hood. With cafés and restau­rants expand­ing out­side and alter­ing the usu­al pedes­tri­an lay­out, she found her­self con­tin­u­al­ly bump­ing into unex­pect­ed obsta­cles and peo­ple. As well as the frus­tra­tion of hav­ing to cre­ate a new inter­nal map of the area, she wor­ried that peo­ple might become annoyed because of her lack of social dis­tanc­ing.

At the same time, she also felt a new threat from peo­ple invad­ing her per­son­al space:

I remem­ber stand­ing out­side a super­mar­ket, wait­ing for my hus­band, when some­one tapped me abrupt­ly on my shoul­der and asked where the near­est car park was. Real­is­ing he had touched me was a shock and made me feel so uncom­fort­able. I asked if he was social­ly dis­tanc­ing and he replied that he had been try­ing to attract my atten­tion for ages. Until that moment I was total­ly unaware he was there.

Conversations with a quantum physicist

Before the pan­dem­ic took a grip of the world, much of my research was focused on pix­els. In par­tic­u­lar, how these tiny areas of illu­mi­na­tion join forces to cre­ate an unin­ter­rupt­ed expe­ri­ence of film with­out ever reveal­ing them­selves – each under­go­ing a dif­fer­ent rate of change depend­ing on the codes they receive.

This led to some fas­ci­nat­ing con­ver­sa­tions with a quan­tum physi­cist, Daniele Fac­cio from my university’s physics depart­ment, about how new tech­nol­o­gy might reveal hith­er­to imper­cep­ti­ble light phe­nom­e­na. His team were using sin­gle-pho­ton cam­eras that can detect light waves as par­ti­cles and thus “freeze” light in motion, tak­ing pho­tographs of a light pulse or video of light as it moves through a room.

As a video­mak­er, I found this tech­nol­o­gy fas­ci­nat­ing – and I won­dered if we could pool our knowl­edge to help blind peo­ple “watch” mov­ing images by trans­lat­ing them into a tac­tile expe­ri­ence. In oth­er words, devel­op a plat­form that could work as a form of “video Braille”.

In 2019, we began exper­i­ment­ing with ultra­sound tech­nol­o­gy to focus sound­waves and cre­ate pres­sure spots that could be felt on someone’s hands. In this way, we hoped we could turn pix­els from mov­ing images into a range of tac­tile expe­ri­ences linked to a film’s con­tent (e.g. facial expres­sions, emo­tions, move­ment). The tac­tile sen­sa­tions could include dif­fer­ent tem­per­a­tures, pres­sures and move­ments on the palm of each hand.

Then the pan­dem­ic inter­vened, our project was put on hold, and time slowed to a frus­trat­ing crawl. A sav­ing grace, though, was my grow­ing under­stand­ing of the way deaf­blind peo­ple take such care to under­stand their sur­round­ings, nev­er rush­ing the process of learn­ing about a new sit­u­a­tion. This helped me to slow­ly accept and learn from this extra­or­di­nary peri­od, rather than try­ing to escape it.

Once lock­down end­ed, I tried to con­vey this by film­ing Issy in her kitchen as she made a cup of tea and arranged a vase of pur­ple flow­ers. What to sight­ed view­ers might look like “fum­bling and stum­bling” (as Issy calls it) is actu­al­ly her way of learn­ing and know­ing. We see her gen­tly touch­ing the flow­ers, smelling their scent, imag­in­ing their forms as she mea­sures their length, cuts and care­ful­ly arranges them into a vase. She is tak­ing as much time as her touch needs:

Although the way I move around might look to you like a strug­gle, it’s not. I am putting my hand out to reach and touch things, pick things up, make sense of what’s in front of me, because that is the way I inter­act with my world. I am draw­ing up a map in my mind of what’s out there. So instead of think­ing I am strug­gling, let me fum­ble and stum­ble – that is all infor­ma­tion for me. The reward I get is that I will be, and am, a much more autonomous and resilient deaf­blind per­son.

A tool to help deafblind people

The insights offered by Issy and our oth­er deaf­blind col­lab­o­ra­tors dur­ing the ear­ly days of COVID made us deter­mined to devel­op a tool that could help give them some inde­pen­dence in nav­i­gat­ing the new­ly opened-up spaces after lock­down. This shift­ed our atten­tion from devel­op­ing a video Braille tool to one that could accu­rate­ly locate the peo­ple and objects around them.

The syn­er­gy we’d already found between arts and quan­tum physics result­ed in our idea for a new “spa­tial aware­ness” tool. Over a series of work­shops start­ing in June 2021, Issy and John helped our research team to under­stand how deaf­blind peo­ple imag­ine, mem­o­rise and map a space both with and with­out touch – and thus what they need­ed from our device.

The pro­to­type con­sist­ed of two ele­ments: a portable radar and wear­able feed­back devices (a head­band and an arm­band). “I am going to be hon­est and say I felt like the borg from Star Wars,” recalls Issy, our first tester. “But wow, it was fas­ci­nat­ing.”

The radar would scan the space up to six metres in front and to each side of the tester, track­ing peo­ple as they came into range and moved about. This infor­ma­tion was turned into vibra­tions of dif­fer­ent inten­si­ty using tiny coin vibra­tion motors in the head­band and arm­band, which acti­vat­ed depend­ing on the dis­tance and direc­tion of the detect­ed per­son.

In our first test in a large the­atre room at the Uni­ver­si­ty of Glas­gow, Issy – hav­ing turned off her hear­ing aids to avoid get­ting any oth­er envi­ron­men­tal clues – was asked to indi­cate the direc­tion of a per­son enter­ing the near-space in front of her based on the vibra­tions she felt in the head­band.

Most of the time, with­out hes­i­ta­tion, she cor­rect­ly indi­cat­ed where they were stand­ing. It was an emo­tion­al moment for her, and all of us, when we told her about the accu­ra­cy of her answers. For the first time since she went com­plete­ly blind, she was sens­ing where peo­ple were with­out rely­ing on touch:

Good­ness, it would be so nice to walk up the road with this tech­nol­o­gy. Along with Yang my guide dog, I’d have a device that can tell me much more about the space around me and what’s hap­pen­ing – you know, how many peo­ple are in front of me, to the side, where are they? Am I walk­ing right into a big crowd?

In our sec­ond test, Issy used both the head­band (to indi­cate the person’s direc­tion) and arm­band (for their prox­im­i­ty) – but strug­gled to cor­rect­ly detect how far away a per­son was. After a few tri­als, we realised the coin vibra­tions motors were too close togeth­er for her to dif­fer­en­ti­ate the sig­nals, and that the fore­arm loca­tion was also con­fus­ing. It would be bet­ter to com­bine the two sets of infor­ma­tion (dis­tance and direc­tion) into one head­band, and use the inten­si­ty of vibra­tions to indi­cate how far away the per­son was.

After fur­ther tri­als, we refined the tool enough to be imple­ment­ed into a cap. From the out­set, our par­tic­i­pants had stressed the impor­tance of cre­at­ing wear­able tech­nol­o­gy that could blend in with every­day cloth­ing if it was to be of true ben­e­fit to users such as Issy:

The fact that it could give me an extra sense of my sur­round­ings is fas­ci­nat­ing. I actu­al­ly just want­ed to say to the guys: ‘Do you fan­cy going up Great West­ern Road with it now?’

‘A magic that reveals the joy in the world’

In May 2022, I was giv­ing Issy a tour of our Touch­Screen event at the Cen­tre for Con­tem­po­rary Arts in Glas­gow. She was imme­di­ate­ly drawn to a video instal­la­tion called Trees, by Wolf­gang Weiled­er. The video shows trees in dif­fer­ent loca­tions being cut down.

While stand­ing in front of the large screen, she said she could sense the trees in the video via her cane. The sound fre­quen­cies from the audio were trav­el­ling from the speak­ers through the ground – she was thrilled because she felt includ­ed in the expe­ri­ence of the art­work.

As we stood there, I shift­ed my atten­tion from see­ing to feel­ing with my feet – and I could sense the vibra­tions too. This new lay­er of expe­ri­ence had been imper­cep­ti­ble to me a moment ago, yet now I felt phys­i­cal­ly relat­ed to the trees as they were being cut down. I also became aware of the ground con­nect­ing me with Issy. The sound was touch­ing us both.

Favour­ing vision over oth­er sens­es means we risk miss­ing out on a host of rich expe­ri­ences and con­nec­tions – not least with peo­ple like Issy, Roger, John and oth­er dif­fer­ent­ly-abled peo­ple.

So the ambi­tion of our ongo­ing research – com­bin­ing deep­er under­stand­ing of the needs of deaf­blind peo­ple with cut­ting-edge quan­tum tech­nol­o­gy – is not only to enable deaf­blind peo­ple to play a big­ger role in soci­ety. We also want to use their unique under­stand­ing of the world to enrich every­one else’s.

There could be more research into tech­nol­o­gy that enables them to com­mu­ni­cate more inde­pen­dent­ly. For exam­ple, by look­ing at how mmWaves (the type of radio waves used in air­port secu­ri­ty scan­ners) could be used to recog­nise hand ges­tures and touch-based com­mu­ni­ca­tion beyond sign-lan­guage.

Cer­tain­ly, there is more for us all to learn about the val­ue of touch in the after­math of the pan­dem­ic. If our eye­sight is about know­ing through a safe dis­tance, touch is about form­ing inti­mate rela­tions and becom­ing entan­gled with the sur­round­ing world. As Issy says:

You know, as some­body who has lost their eye­sight, I was just too busy try­ing to get on with things. You don’t stop for two min­utes and think: ‘Well actu­al­ly, I hadn’t thought … how much I rely on touch and how much it means to me. How much it helps me to visu­alise the world.’

For John, touch is a “holis­tic way of feel­ing” through the body. For Issy it is about “imag­i­na­tion” and know­ing through “fum­bling and stum­bling”. For Roger, touch is like “mag­ic” that reveals the joy in the world.

It is sad that it has tak­en a pan­dem­ic to bring greater under­stand­ing of the sig­nif­i­cance of touch – and in par­tic­u­lar, touch depri­va­tion – in our dai­ly lives. But per­haps the dis­con­nect­ed­ness we all expe­ri­enced has also evoked greater empa­thy for the strug­gles deaf­blind peo­ple have been expe­ri­enc­ing through­out his­to­ry, such as iso­la­tion, lack of effec­tive com­mu­ni­ca­tion and exclu­sion from soci­ety.

It’s time we embraced their unique insights and learn about the way they “see” and feel the world. Or as Issy puts it:

I always say to peo­ple, ‘You come into my space for two min­utes and I’ll show you the way, in my world and my deaf­blind cul­ture. The way I inter­act and con­nect with my space. Walk with me and I’ll show you the way – not through your eyes … but by con­nect­ing with me and my hands through touch.

This arti­cle is part of an Insights series devel­oped with UK Research and Inno­va­tion (UKRI) to explore the wider impli­ca­tions of research car­ried out dur­ing the COVID pan­dem­ic. Touch Post-COVID-19 is a UKRI-fund­ed inter­dis­ci­pli­nary research project based at the Uni­ver­si­ty of Glas­gow.