Hybrid Mixed Reality

Nicole Yu
15 min readDec 2, 2020

December 1, 2020

Interviews and Personas

For the interviews, I talked to an interesting variety of people. I actually didn’t finish reading the project prompt before I started asking people to interview, but I think it worked out alright, and it gave me a good excuse to catch up with remote friends. I also think I gathered a large spectrum of perspectives. The people I talked to were Grace (all remote), Zimmy (hybrid), Stanley (a remote MCS friend), and Josh (a remote archi kid).

I started off by asking what motivates the interviewee to work, since based off of personal experience, I find it extremely difficult to focus in an environment I’m too comfortable in (probably why I didn’t finish reading the project before starting). In general, design students found that being able to work in an environment with like-minded people, and being able to look over and bounce ideas off each other as really beneficial. For the people outside of design, it seemed that time was the biggest motivator, whether it be just simply having too much, or having too little as a result of procrastination.

I then asked what they wished they could get out of studio/school life but can’t due to being remote. A lot of people talked about the social aspect of studio/life, how it’s been really hard to meet new people/talk to less familiar classmates, and how there’s so many barriers when socializing is usually so effortless. However, despite that, the non design kids found that games helped them bridge the gap easily, and as a result they didn’t feel as lonely. But one interviewee did bring up the fact that games sometimes are just a way to escape reality, which isn’t healthy, despite the social benefits.

When I asked about they felt about the classes, most design students were pretty content with the set up, with one mentioning that the professors could be more lenient with their expectations for our work and work ethics, but that it definitely isn’t sustainable, especially in design, given how important our portfolio is. However, the non design students had much more grievances about the learning experience, which makes sense. They mentioned how professors spend more time making sure the students can appropriately get the information they’re talking about, rather than answering questions (which is hard to do over Zoom). That’s because they have trouble making sure that the camera can see the whiteboard clearly. Also, due to the remote set up, since it’s harder to participate, it’s harder for the teacher to notice you, which might take a hit if you decide to ask them for a rec letter for grad school. The archi student mentioned that they moved a lot of stuff to 3D modeling, which makes him miss out on being able to hone his physical modeling skills.

Another thing that came up was how to update progress work. For Grace, being able to see what others were working on was really helpful in gauging what was going on, and how far long she was. Unlike those in studio, she didn’t have the luxury of casually talking to others and walking around to see what others were doing. However, some people wouldn’t update Medium appropriately, so it was difficult to gauge. Zimmy also mentioned that seeing what other people were doing definitely helped her a lot, but for Medium it’s definitely a different experience with its own barriers. Here, you feel obligated to say something, since the professor will read it, rather than the effortless conversations in class. Grace mentioned that if Medium updated like how Google Docs did, it would be really helpful.

There are more questions I asked, but you can see bits and pieces of the answers in the persona spectrum I made. Overall, I think the biggest issue I see are the amount of thresholds people need to go through to do something that used to be second nature. That’s what prevents people from updating Medium regularly, participating in class, reaching out to “strangers”, etc. I think I might focus on something that smooths the whole experience down.

Follow Up Interview:

I asked my friends what made them constantly go back to their games, and they said that it’s the fact that all their friends play it (they wouldn’t play it alone), and also being able to upgrade skins and characters keeps them coming back.

I might add some interesting incentive?

Personas

I made personas later, after I got a sense of what I wanted to do. I focused on the differing workflow and needs of these three people (I tried to make them totally different). I think this was really useful, since it helped me think about what needs were similar between them and try to brainstorm something that would work out.

In Class:

Peter’s suggestions to look into:

  • Mercury os
  • Makespace.fun

December 3, 2020

Workspaces — Digital vs. Physical

Looking at my digital and physical workspaces, I found that a trend I have is that I keep “official” stuff clean and organized, and the rest is disorganized, but easy+quick to access when I need it. By seeing my habits, I can see why I’m drawn to creating an effortless experience, given how lazy I am with keeping structure. I also need a “closet” to stuff miscellaneous items in to keep everything clean. (I clean my computer desktop after I finish projects by putting them in folders, and I clean my desk by putting everything onto the second desk)

Programs I Use:

Social (specifically to talk to other people): (Roughly ranked)

  • Facebook Messenger — I think most people have a Facebook, and as a result, this is the easiest way to find and reach out to people, and typically the best place to make group chats. Personally, I like using this more than text messages, since it’s consistent across platforms, since weird things always happen when, let’s say, you try to send things from and iPhone to an Android. Also, group chats feel a lot more organized here, and the features like reply and being able to name the group chat are really nice.
  • Snapchat — Not a lot of people use this, but personally, I like using Snapchat. Compared to Instagram, there’s a lot less pressure to perfectly edit a picture, which takes time and brainpower. Also, the casualness of it makes it easy to subtly check in on your friend’s lives, since pictures can say a lot.
  • Discord —The thing I like most about Discord is how you can categorize your group chats. Especially when it comes to things like group projects, it’s nice being able to easily find past conversations based on category, or talk in a channel without alerting the whole group.
  • Messages(Apple text messaging) — I mainly use text messages when I need to immediately get someone’s attention or for people who don’t have social media (like my parents). I think most prefer other platforms due to how finicky text messaging gets.
  • Zoom — I only really use this for class. I think this is the least personal out of the social programs I’ve mentioned

Other Programs:

  • Google Chrome — For the internet, but I also like for the useful extensions and how all your bookmarks are saved to an account, so you don’t have to rebookmark things when you go to a new computer. It also makes it easy logging into any Google program
  • Adobe Creative Cloud — Really useful for projects
  • Figma/Miro — Good collaboration platforms, and I like the free format, like pinning stuff on a wall
  • iPad drawing apps — when explaining concepts, I find that TAs are much more comfortable when they have an iPad (acts like a whiteboard) compared to the ones who try to awkwardly write on a trackpad or typing. I’ve also found it useful when I want to annotate pictures
  • Entertainment(Netflix, Spotify, etc) — used for procrastination or just make the work environment a little more interesting

AR Experimentation:

For Fun

These are some interesting experiments I made when playing around with Reality Composer. The first one is the one I made in class. I think it was pretty fun seeing how the elements interacted with each other.

Interesting Interactions

Here, I started to play with some actual user interaction. The first idea I played with was using things like marbles that cause something, instead of using normal buttons. Though it’s pretty cool, it’s kind of impractical and sometimes unpredictable. If the marble were to ricochet at an odd angle then the interaction wouldn’t be usable.

For the second one, it was pretty simple, just a ticket machine that gives you a ticket when you get near.

Possible Interaction

Here, I tried to sort of apply my work habits into an AR setting. I like having everything laid out before me but still somewhat organized. I found that that could sort of be done, through dimensions, important things would come closer to you, and less important things would be farther away. “Sub” files would be the next step, as seen in the top left corner. At the end, you can condense everything into one when you don’t need it, and the next time you need to use it, it would be in the same order.

  • Extra note: I noticed that this time I was working at my dining table. That’s because this location has the best lighting out of all the places in the room, so sometimes I work here. It’s also the cleanest “desk” in the room.

After all this, I think I want to make a collaborative space used to organize and make sense of ideas and is effortless to use.

Possible Idea:

Thinking about it, should also have an mode where you just sit on the desk and navigate with movements on desk or something, for lazy days

Self Reflection

As the prevalence of digital media in our physical environments increases daily, what is the role and/or responsibility of designers in shaping our environments?

As digital media in the physical world increases, the biggest issue becomes accessibility. The older population already finds it difficult to use current technology, imagine something that deviates even farther from current technology. Someone in my family gifted my grandma a laptop, but I’ve noticed that she doesn’t really use it much, and there’s so many reasons why. First of all, the keys are silver, backlighted in white light, so it’s hard to see the keyboard. Then there’s the laptop’s UX, before, she was using a 2007 desktop, and so much has changed since then. Having to relearn everything must be so difficult if it’s not already intuitive.

Additionally, there comes the price. New tech always comes at a hefty price tag, and as a result, only the well off can really afford it, and if this new tech helps boost productivity, then the gap between the rich and poor will just widen. It’d be pretty important to try to cut costs without sacrificing quality.

Additionally, when it comes to tech, there’s safety issues, namely hackers. This may not be the designer’s job, but they should make sure that they’re designing for a company that can ensure the safety of their users, since as the two worlds merge, people will probably put more of them selves in the digital world, making them more vulnerable.

Feedback Notes:

  • Project room, dedicated, much more effective
  • Can put process up, more grounded
  • Each class has a cloud, bring up contextual info, links to website, products
  • See everything at once
  • What do you want to see the same across all
  • Reduce interrupt
  • See coming towards you, footsteps, social cues, how to create cues like that, and can see when people show up, how to navigate
  • How to start interaction
  • Laptop video feed, on an avatar, representation
  • Decide, when valuable to see, and when ok to represent
  • Start storyboarding on post it notes

Next Steps:

  • Make an interaction in Reality Composer
  • Storyboard interactions
  • Figure out how to represent people
  • Figure out logistics of how it works

December 8, 2020

Prototyping

I want to create something that people can use as another workspace to organize their thoughts and easily share with other people. This is my Reality Composer prototype, it starts off as a “folder” that expands into your last saved workspace. Then, you can bring over something that you need to your computer and transfer data, etc, whatever you need to do with it. Right now the interaction feels kind of one sided and clunky, and I want to find a way to make it more collaborative, like using sticky notes, drawing things, some kind of notification thing that shows someone’s there, how to represent people, etc.

(For fun)Experimenting with Scanning Things w/Lidar

My iPad has Lidar, and I was hoping that I could use this to make AR things in my video. It would be a good way to show how you can scan your model and put it in your workspace, but the technology isn’t really there yet (but probably will be in the future). I find it interesting how it did a better job at scanning large things rather than small things, as seen with the deformed mouse vs. the desk space.

Feedback Notes:

  • 5G can help with tracking movements of the avatar
  • Avatars can look realistic (maybe tech will be that advanced in the future)
  • How to notify people are there (like in Figma?)

December 11, 2020

Storyboarding

For the “storyboard”, I basically sketched out the interactions that I wanted to happen, in to particular order. My goal was to create a workspace with an adequate amount of tools that help collaboration. On the system diagram I have the Hololens/user in the middle, and what contributes to this experience. I think a big thing for me is easy sharing through devices and being able to save things onto the cloud easily. I think this would help a lot with making the whole work process smooth and effortless. I also added a stylus, since that would be helpful in precisely pointing at things and drawing things, since that’s usually awkward when just using a finger.

Feedback Notes:

  • Main thing: too many ideas, try to reel it into a cohesive story
  • Possible idea: Galaxy like — orbiting circles?
  • Stylus — possible 3d drawing issue — maybe sticky note appears when you look at a flat surface (2D surface)?
  • Bubbles bump into each other when you merge
  • Think about how it interacts w/ physical environment

December 17, 2020

Refining Ideas + Final

To help make my ideas less scattered, I decided to write a list of what would happen if the user were to use this workspace to do their work. This way, I could make a story, and see what ideas would fit, and what ideas wouldn’t. I also decided to use After Effects, rather than Reality Composer, since I’d just have to match up the object with the actor’s hand, rather than try to get the actor to put their hand in the right place when they can’t see what’s going on. I then created a storyboard to really visualize the shots I wanted to make. I kind of wanted to emphasize how the collaboration works despite the different locations, which is why I color coded to boxes red and blue (red = at home, blue = at studio). This was helpful when actually shooting the video, and I only had to reshoot something once, and that was because the angle was just really off and weird.

It was definitely an interesting experience juggling After Effects and making the story make sense.

Presentations Reflection

After seeing the other presentations, I found that I think I got the “pitch one idea” part wrong. I feel that my idea was too broad/a mix of a bunch of single pitches even though I thought it got simplified down. Compared to the others, it felt a little chaotic. If I were to do this again, I think, actually, I would just focus on the bubble idea. This idea was created really late into the process, but I found that idea the most fun and interesting. I think I would refine that down better by maybe have distance play a role as well, maybe you can faintly hear conversations when your bubble is farther away, similar to in real life. Maybe there’s a funny animation when someone rejects your call. I also just realized that this gives another meaning to when people say “get out of my bubble”. Regardless, this project was fun, and as you can see from above there’s so many directions you can take, which I really liked, even though it put me off track a little.

Self Reflection

How were the skills you developed in the first project similar and/or different from the second project? What is your understanding of the role of an Environments designer?

For the first project, I felt that it mixed skills learned in the P mini with E, and the second mixed C and E. As a result, I was much more comfortable when doing the first project, since I felt that all I really had to worry about was the E part. On the other hand, for the second project, I had to worry about both aspects equally, since I struggled a bit with After Effects, and spent a lot of time googling and asking around. I don’t think this is bad though, I’m glad I got to hone both my skills in C and E.

I think in the first project I thought more about the physical space and could actually “walk around” and sort of feel it out, and I was confined to that, but in the second project, there were an unlimited amount of possibilities, and I couldn’t help myself from trailing off into different possible directions. In the first project I learned to cut down with space in mind, and in the second I cut down due to just not being able to get to them all.

For how my E skills developed, I think for the second project, it got easier to envision the interactions the user would have, rather than in the first project, where I was still unfamiliar with the concepts. I think E taught me a lot about the importance of having a story for a product/space being used. It’s helped me really understand and flesh out the product/space. Before, I just designed things just for one use case, but now I feel the need to think through the whole scenario, giving a more through and deep solution. I feel that my previous designs were more shallow.

From my experience in this class, the definition of an Environments designer has become slightly clearer, but I think that’s ok, since the lines blur so easily for what Environments design is, especially with the improvements in technology. I think in the past, it was like our Miller project, designing how people interact and feel in a space, but now, this “space” can apply anywhere, from app interfaces to AR and VR, and literally any other future tech that is going to be used by people. I think what Environments designers do is bring together everything (literally everything, I think they have to have a lot of knowledge and keep up to date with what’s out there) and improve/create the experience of using a product/space, compared to C and P, which create the product.

--

--