056: Leveraging LLMs: Britney Muller’s Insights on AI, Advocacy, and ASL
Join us for a captivating conversation with Britney Muller, where she delves into the realms of mentorship, AI innovations, and her enchanting ASL escapades. Explore the intersection of expertise and experience in this digital marketing adventure!
Key Points + Topics
- [01:18] Britney Muller originally attended the University of Minnesota to study medicine. After a year at school, her family persuaded her to switch from Pre-Med to Journalism. They saw that her interests and passions around marketing and writing would be better fulfilled down a different path. She learned a lot outside of the classroom. She “invented” a couple of jobs for herself. She really wanted a yoga membership, but it was expensive, so she offered to become their social media manager for a membership.
- [04:55] Over time, Britney has grown to value relationships on a deeper level. While these relationships may appear to be mentor relationships, Britney views them more as friendships. Mentorship conversations happen, but they’re not the whole of things. The digital world really offers the opportunity to reach out and connect with others in a way you wouldn’t be able to otherwise.
- [07:20] Britney stands on the shoulders of giants. She was once writing real estate listings for houses in Breckenridge, CO. Peter, the realtor she was working with, would drop her off at the huge, fantastic homes, and she would write listings while spending time in the homes. This realtor was very action-oriented and supportive of Britney’s career and noticed early on how uncomfortable she was speaking about money. He helped give her the space and guidance to learn how to do freelance work. After Peter influenced her career, she found herself connected with Rich Stats in Vail, CO. He’s one of the most brilliant people she’s ever met and is constantly thinking twelve steps ahead of everyone else. He has such powerful marketing ideas. He helped Britney learn what she was good at and what she wasn’t. Through their guidance, Britney found herself late paying rent one month because she knew she had to attend MozCon. If you can encounter a positive mentor and match that with some action-oriented effort, that can be incredibly powerful.
- [11:25] Speaking of MozCon, Britney later found herself working at Moz and with Rand Fishkin. Britney will always sing his praises. She knows, even though it might shock him a bit to hear, that he has made her a better human. He’s taught so much more than he realizes. She knows it was a bit like working with a celebrity and can recall the first time she had to disagree with him. But he’s so good about creating safe spaces for disagreement, discussion, and learning. Rand Fishkin is a legend.
- [14:35] These days, Britney mentors a few individuals, but not in a formal sense. She checks in with them regularly on their work, goals, and lives. She will regularly pass opportunities on to them. She was recently offered a speaking opportunity in Australia that didn’t make sense to her. She immediately thought of a woman she mentored in New Zealand who would be perfect for the talk. Britney knows how powerful it is to be so thoughtful and intentional about introducing people.
- [16:13] What are Large Language Models (LLMs)? They are VERY LARGE, deep learning models pre-trained on vast amounts of data. Though Britney knows we don’t truly have artificial intelligence (AI), and it’s possible we never will, there are two types of “AI,” as the world uses the term today.
- Generative AI: creates new outputs based on user-supplied inputs.
- Deterministic AI: this analyzes and identifies information, such as labeling images and identifying cancer in medical imagery.
- [17:28] – Britney sees many marketers going to generative AI for deterministic tasks. It’s incredibly important NOT to be fooled by LLMs as they’re very clever and confident sounding, but they’re not always accurate. They’re basically giant word-guessing machines. They’re estimating their next word and answer based on large amounts of training data. They tend not to be great with industry-specific research. Britney would NOT recommend using AI or LLMs for keyword research; it just results in word garbage. However, if you already have your keyword lists, deterministic AI can be very effective at grouping these keywords.
- [20:20] – Britney attended NeurIPS recently and was blown away by all she saw and learned. Neural Information Processing Systems (NeurIPS) is an eight-day long conference with dozens of talks, often thirteen, happening concurrently. She was very surprised to see how much disagreement there was among those at the top of the field. She had expected to leave the conference with clear, solid answers but instead left with more questions and concerns because of all she learned. Britney’s predictions for AI, LLMs, and the future focus on trends she sees today. As many of the largest companies in the space, controlling the largest LLMs are public companies, it’s in their best interest to maximize ROI and profit. This means they will likely be rolling out applications and integrations on a grand scale in ways that might not make the most sense, just to see what sticks. She is also beginning to see small startups of hyper-industry-specific LLMs, think medical, finance, law, etc. There are many potential applications in those spaces to save time and resources with very fine-tuned, domain-specific LLMS connected to a database of relevant, accurate information.
- [23:25] One of the core disagreements among the very top minds in AI consists of Yann LeCun, Geoffery Hinton (the “Godfather of AI”), and Yoshua Bengio. LaCun believes that since humans still control the connection between AI and ACTION, there’s no real risk. Humans can always look at the data AI presents and decide it would be foolish to enact that data. He believes the fear-mongering is more about consolidating power (and, by extension, money).
- [25:30] One day of the NeurIPS conference was set aside for various affinity and minority groups to give talks about their research. Britney was very disappointed to see these talks sequestered away from the main floor in small rooms at the same time as very big talks from big names in the industry. This led to these talks being very poorly attended. The speakers were all so proud to be there and present their research. Black in AI, Latina, and Indigenous in AI got no visibility even though they were discussing things like the loss of languages across the globe. To hide these speakers away also goes against everything we’ve statistically proven regarding diversity. Greater diversity in data sets improves the outcomes of models. Britney even wrote a paper about the vast amount of information missed by other conference attendees failing to listen to these affinity groups.
- [28:45] Given that the training material and coding for LLMs and AI are created by humans, does Britney believe it’s possible to remove bias from LLMs eventually? Not really. There will always be biases and human error, but as we learn more about these inherent biases, we can better mitigate them. One of the ways to mitigate biases is transparency. Wikipedia is incredibly biased. But now that we know the specifics of those biases, we can better adjust and navigate them. It’s the closed, black box systems that share no data on what they were trained on that create bigger problems. Some LLMs are trying to mitigate errors with multiple agent LLMs.
- [31:10] There’s a funny problem in AI – Giraffing. AI has a tendency to interject and identify giraffes at a far greater rate than they exist in reality. Because these LLMs are trained using data from online, it has received a disproportionate amount of giraffes in its training materials. In a similar vein, Britney once worked with a company for image generation AI. If this imaging engine was asked to generate an image with no other prompt, nine times out of ten, it would produce the image of a woman in a sari. This just goes to show that LLMs and AI are foolish in ways we cannot anticipate. And it can lead to dangerous influences on people’s worldviews.
- [32:38] OpenAI, the owner of ChatGPT, is currently in litigation with the New York Times regarding copyright infringement based on the data OpenAI used to train its LLMs. Britney recalls a similar case (on a much smaller scale) where a small publisher won a lawsuit regarding the use of the material and subsequently received royalties any time it was evident their content had been used in generative AI. Many legal scholars also seem to believe many of OpenAI’s arguments won’t hold water. Britney is interested in seeing the outcome of the lawsuit and believes it’s likely to set an industry-wide precedent.
- [38:10] Britney will always be a proponent of high-quality content. When arguing quality over quantity, she has a few case studies ready to go at any point. Many of them come from content strategy with a high time investment for incredibly rich content. All of her marketing gems have stood the test of time because she was able to truly dive deep into a specific topic and deliver long-form, high-quality content. She still gets calls from clients from whom she wrote content eight years ago to rave about how the traffic has snowballed since first posting. Britney does believe that internet marketers as a whole have gravely undervalued public relations (PR) and it’s a tool she’s been using and enjoying a lot lately.
- [41:31] It all started with a middle school crush. Britney had a crush on a deaf boy when she was in middle school. In order to learn sign language to talk to the cute boy, she convinced his mother to enroll the boy’s brother and her in an American Sign Language course. This led to a lifelong love of the language. When she travels, she will often try to connect with other deaf communities and share in their distinct signed languages. When she was younger and very into skiing and snowboarding, she found herself needing a lift pass (that wasn’t cheap). So, she offered to be an ASL interpreter for ski lessons. So, she would teach snowboarding lessons as well as interpret for other instructors on the snowy slopes.
Guest + Episode Links
Danny Gavin 00:05
Hello everyone, Danny Gavin,founder of Optige, marketing professor and the host of the Digital Marketing Mentor. Today we have Brittany Muller, founder of DataSci101. Brittany is an international keynote speaker, writer and SEO consultant. She’s passionate about original research, machine learning and automating workflows. Today she’s founded her own company, DataSci101, where she strives to make machine learning and AI accessible for all marketers. Today we’re going to delve into large language models, generative AI, world-class content and a whole bunch of other fun things. How are you doing, Brittany?
Britney Muller 00:58
I’m doing well. How are you, Danny?
Danny Gavin 01:00
Good, I’m doing really well. So we bumped into each other. At Brighton SEO a couple months ago, I got to hear you speak and I’ve been following you for a while, so it’s really a big honor to have you on the podcast today.
Britney Muller 01:12
Thanks for having me. I’m so excited that we’re able to do this.
Danny Gavin 01:17
So let’s first talk about where you go to school and what you study?
Britney Muller 01:19
Yeah, I went to the University of Minnesota and I actually started out pre-med and then, after about a year, decided to make the switch to the School of Journalism, where I studied strategic communications, public relations.
Danny Gavin 01:37
Awesome, and that doesn’t surprise me that, like Brittany, liked pre-med right.
Britney Muller 01:41
I love it, yeah.
Danny Gavin 01:43
What caused you to make that switch?
Britney Muller 01:45
You know, what was funny was that there was a lot of persuasion from my family. So I think they kind of saw very early on that I was looking towards a career, as they put it, like in a basement somewhere, like tucked away in a lab, which quite frankly I know I would have loved it. So there’s nothing like against that there. But they, you know, they saw these other interests and passion of mine, particularly around marketing and writing. I’ve always been a writer and so they kind of pushed me to explore something in the School of Journalism and, yeah, very happy that they did.
Danny Gavin 02:27
So, when you look back at your time at the University of Minnesota, are there any experiences both inside and outside of the classroom that you feel were impactful in directing your path and where you are today?
Britney Muller 02:36
You know, what’s funny is I feel that I learned more outside of the classroom than I did in, particularly in my career path. I was just scrappy because I had to be. You know, as a college student, I remember desperately wanting a yoga membership at was like the popular hot yoga place. You know, I couldn’t afford it and so I invented a job where I would be their social media manager and I would do all of these things like marketing things for them and they let me do it. And I got free yoga. And similarly, I walked into a couple other companies and was like, hey, I’m going to. I pitched to do this thing, here’s how I’m going to do it. So I think it just gave me a lay of the land as far as, like, working with people and interacting with people. I think waitressing was also a huge part of that. Like I very quickly learned how to navigate different kinds of people, how to read people, how to react to different negative situations, and all of that has sort of prepared me for the career I have today.
Danny Gavin 03:43
So going up to someone and kind of bartering, or even I mean being a waitress, where do you think that comes from? Like, is it just you, your personality, or were there certain things that your parents or liked when you were younger? Was there something that instilled that in you?
Britney Muller 03:57
Yeah, oh my gosh, Daniel, it’s such a good question and I actually have thought a bit about that and I think there’s some like parental Minnesota culture baked in there, Like, first of all, Minnesotans are just like good, traditionally good people, but we also have this like probably unhealthy amount of like wanting people to like us right, Like it’s. It’s baked into our DNA, Like we really want people to think we’re good people and, you know, like us. And so I thought very early on I identified that in adults around me and I navigated that as like a child and so coming up into, like having my first waitressing job. Some of that came more naturally to me than maybe it might others.
Danny Gavin 04:44
Oh, that’s so fascinating, they’re great. Be cool If we should do it. It would be cool to do a little study, right yeah?
Britney Muller 04:49
Just.
Danny Gavin 04:50
Minnesotans in general are right.
Britney Muller 04:51
Yeah, absolutely.
Danny Gavin 04:53
Cool, so let’s dive into mentorship. So how would you define a mentor?
Britney Muller 04:57
So I have a pretty possibly unique way of identifying mentors and that’s less in the formal sense of you asking someone to be your mentor and they, you know, provide you with support and feedback. I, over time, I have really learned to value just relationships in general that look might look a little bit like mentorships, but they’re at its core they’re friendships, right, they’re kind of there’s a two-way interaction there that is mutually beneficial, mutually supportive, and you know, oftentimes one person has just more natural insight into a particular thing that you might be wanting to accomplish or do and that’s where some of that like more mentorship conversation come, come about. I guess.
Danny Gavin 05:48
Yeah, and I think in our industry in general, having colleagues and just having peers, you know, it’s an environment where, yeah, I’m not saying everyone’s good, but there’s a lot of good people, a lot of people who want to connect and offer. So I don’t know if it’s like that everywhere, but I feel like, specifically in digital marketing, it’s a lot easier to have, like those mentors, slash friends, slash colleagues.
Britney Muller 06:09
When I was younger and even just out of college. You know you have a lot of advantages as far as, like, trying to connect with people digitally right, reaching out to people and doing it in a way that communicates value. It communicates that you’re maybe scrappy and bringing different things to the table. You know what I mean. Like I think there’s way more opportunities available for people to connect with others they feel like might be outside their reach and oftentimes we just don’t even try right. So I think to give yourself the grace of just trying to connect with some of those people that maybe you admire, you follow, and doing it in a way that’s not, at least in my opinion. That’s not just hey, will you be my mentor, it’s more of like a value driven conversation. You know, maybe, hey, I looked at all of Danny, your incredible podcast for this last quarter and I automatically transcribed them all and did that. You know, bring value. That’s interesting and that can kind of plant the seed of that relationship.
Danny Gavin 07:15
Yeah, I love that. So let’s dig into some of your most influential mentors. I know some of them are like rock stars. I don’t know all of them, but let’s talk about them and how they influenced you.
Britney Muller 07:26
I feel like I stand on the shoulders of giants, like they’ve influenced me immensely and starting like very early on where I connected with a local realtor in Breckenridge, colorado, who really took me under his wing and he would, he was so sweet, he would drop me off at these like huge houses in Breck and leave me there for the afternoon. I’d bring lunch and I would just get to like being in the house and write the listing and that was really cool, like trying to incorporate, you know, the Breckenridge culture that had come to like love and know so well into an actual mini side job. For me it was so fun and he, along the way, was very kind of action oriented and supporting the growth of my career. And he picked up very early on how uncomfortable I was. Talking about money, I mean, I would just like I would melt, I felt like dying right, like coming again. This is some like Midwest roots there or like you don’t talk about that, you know it’s inappropriate blah, blah, blah. And he really gave me kind of the space and made it comfortable for me to just start to learn the ropes a bit when it comes to doing freelance work, like writing and different things. So that was monumental for me. And also, you know, he paid for my first SEO class and I still have those notes in a frame. They’re in storage but it’s so funny, you know, it’s bold keywords, it’s all this stuff. So he was absolutely influential and that was Pete dining or that’s his name, but yeah, he’s still out in Breckenridge.
09:01
And then, after Pete, I connected with Rich Stats in Aville, Colorado, who, oh my gosh, I love so much. I consider the Rich family. He is one of the smartest people I’ve ever met in my whole life and he’s always thinking 12 steps ahead of everyone else and he has some of the coolest and most powerful marketing ideas I’ve ever come across. And so working with him was like I don’t know, it was so much fun, it made this space so much fun. He introduced me to Whiteboard Friday. He gave me confidence. So that kind of support is so important as you’re identifying your skills and learning the craft and getting comfortable with what you’re good at and, equally important, what you’re not good at. So, yeah, also hugely influential.
Danny Gavin 09:55
And you know what I’m not saying. You take it for granted, but I think sometimes people do take it for granted, because sometimes you end up in a bad agency, right, or a horrible situation, and I think for what you’re saying is look at the difference like a horrible situation can really stunt your growth, change these opportunities, but a good opportunity could turn you into a Brittany Muller which is pretty awesome.
Britney Muller 10:18
And that’s all it takes right. It’s like one or two of those and that changes everything. But yeah, you’re exactly right, there are obviously the neutral or the negative experiences out there. But if you’re able to encounter a positive one, and then I would also say, and match that up with a bit of action-oriented effort, I do look back at my transition from working with Rich to going out on my own and I took a ton of risk. I remember I was late on my rent that month, that I went to MozCon for the first time ever because I had to go. There was no other choice in my head. I was like I knew all the speakers that were going. I had followed them religiously. That was. The next step for me was making those connections in person. I had questions for specific people. I brought things for people. I was delusional. This was just my world and I was so excited to be a part of it.
Danny Gavin 11:24
So cool, so I guess that leads on to talk about MozCon. What about your relationship with Rand?
Britney Muller 11:28
He’s the best. I can’t speak highly enough of Rand, honestly, and you know what? I’ve been thinking a lot about it and I think, while there’s all the diamonds, I’ve been trying to craft in my head a way to communicate this to him in a way that doesn’t freak him out but communicates the value that is there. But he’s just made me a better person period. It’s insane. I mean I could list so many things, but it’s just like, oh, that man has taught me so much Way more than he even probably realizes, because I’ve been around him enough to see how he handles things, how he navigates different situations or sticky meetings or this or that. It’s just, he’s one of a kind yeah.
Danny Gavin 12:17
And when you had a chance to work with him, was it like working with a celebrity, or did that wear off soon?
Britney Muller 12:24
No, it definitely felt like I was working with a celebrity. It was crazy. I was pinching myself all the time, all the time, and I still remember the first time I disagreed with him and how just sick I felt. I was like, oh my god. But that’s something that I also love and appreciate so much about Rand is he really creates a safe space for disagreement and he’s one of those brilliant minds that values understanding other sides to the story and where different people are coming from, and so I very quickly kind of got over that. But yeah, he is, oh my gosh, he’s a legend, he’s just an absolute legend.
Danny Gavin 13:10
Let’s talk a little bit about your parents. I know we mentioned a couple of minutes ago, but I know you put them down as your mentors. But when you look at them, what do you feel like? What’s the most that you’ve gained from them?
Britney Muller 13:22
My mom is the most thoughtful person on earth. She would do anything for anyone. So would my dad. They’re just two of the kindest, most beautiful people ever. And they’re also so different. My mom’s very bubbly and outgoing and she can be loud and fiery, and my dad is more quiet but always knows just what to say. And I remember I’ll never forget I was reading this book. I was taking my dad and my brother to Vegas for this work trip. This was over 10 years ago and I was reading a book that was describing cities in the US with one word descriptors. So I want to say Las Vegas was sin and LA was famous and New York was successful, and it was just really cool. And I was talking to my dad about it and I was like, what would your word be? And without any second thought, he was like integrity and that perfectly describes him, like 100%.
Danny Gavin 14:25
And that’s a really, really important attribute to have. So let’s move over to how you mentor. In your current position, I believe you do mentor a few young professionals regularly. Tell me about that.
Britney Muller 14:35
Again, it’s not like there’s no formal mentorship, I would say, but I’m checking in with them. I also like volleying over opportunities when I see appropriate. So one young woman is actually in New Zealand and there was a speaking opportunity in Australia. That just didn’t make sense for me anyways, but I immediately thought of her and what a great step that was, and this period of time for what she was doing, what she was interested in and her education, and so things like that are my favorite things in the world. That’s, making those connections for people and making introductions. I think that’s a big thing that could be considered mentorship that maybe people don’t think about as often. But some of the most influential mentors I’ve ever had, especially Rand, are so generous with his introductions. He’ll bring your name up if it’s relevant. He’s just so thoughtful at connecting people and so I try really hard to do that and do it in meaningful, appropriate ways.
Danny Gavin 15:50
Yeah, and once again you don’t realize. I mean, it’s amazing how you can change someone’s life by doing that, and sometimes people forget that by giving you actually get more. So Totally. And that’s not the reason why you do it Right. I’m not doing it because I want to feel good, but it’s one of the added benefits.
Britney Muller 16:09
Yeah, absolutely, absolutely. I love that.
Danny Gavin 16:13
Let’s jump into your areas of expertise. Let’s first talk about large language models. Let me first define LLMs or large language models. They are very large deep learning models that are pre-trained on vast amounts of data. What are some common ways that we’re currently using LLMs?
Britney Muller 16:31
First of all, I feel as though most marketers are not even thinking about LLMs in the appropriate construct of what it is. So, just to break down what it is, there are two types of quote unquote AI. We haven’t reached AI yet. There’s no such thing. There probably won’t be for a very long time or never, but whatever, that’s what we’re using now. So within these two types, there’s generative, so things that create entirely new outputs like images and text. They’re generative. And then there’s deterministic. These are more of your image labeling, your cancer identification. These are more bought on. You want these to be super, super accurate, whereas the generative is just sort of hallucinating at all times and making stuff up. That alone is super, super important to differentiate, because what I see marketers doing is going to generative AI like an LLM for deterministic tasks and that’s not really what it’s made for.
17:37
It’s important to not be fooled by LLMs. They are very clever and confident sounding and they sound like they’re super intelligent and we need to keep in mind these are word guessing machines. They are estimating their next word off of a giant probability distribution of everything they’ve seen over their training data, and while, again, this looks very convincing and very confident, it is not always the most reliable or accurate. It’s not great with, like, industry specific insights, so there’s different aspects like that. The thing that kills me is when people are using this for keyword research. Right, this is like a word for garbage. This is just, it’s making stuff up and while that’s great if that’s what you want, if you don’t want the data driven strategy and you just want some content ideas, go use it. But if you’re from a traditional SEO marketing background and you want to be really strategic in how you’re targeting your content SEO-wise, I would not use it for something like your research.
Danny Gavin 18:52
What about? Let’s say, a process that a lot of SEOs will do is, after they’re done their keyword research, they’ll group keywords and even in PPC we group. Do you feel like using that tool to help a group? Would that be a good idea or still a bad idea?
Britney Muller 19:06
I love the clustering models. I think they’re brilliant. It allows you to see the forest from the trees and it gives you a bird’s eye view of clusters of keywords and topics that go together. I think it’s incredibly valuable. I hate it now that I’ve been using it for the last several years. I hate just looking at a giant mess of a spreadsheet that isn’t organized like that. So, yeah, that’s a great, great example, but that’s not generative, right? So that algorithm is, that’s more of your traditional natural language processing models. So, again, this is where people get confused as to what LLMs are capable of, which is something I get excited about, kind of educating and sharing the knowledge, because people that are doing their work day in and day out. You’re going to think of the next brilliant application for what you do. So it’s important just to have that foundational awareness of what is it good for, what’s it not good for, fundamentally, how does it really work? And that information essentially teaches you how to fish moving forward.
Danny Gavin 20:16
So what are your predictions for the future of LLMs?
Britney Muller 20:19
It’s wild. So I don’t know if you saw this, but I went to NURBS last month, which is one of the number one AI conferences. 16,000 people it’s crazy. All the big teams are there. There’s over 13 talks going on at once. It’s just an eight day thing. It was hectic.
20:38
What I was surprised by was the amount of disagreement between the people at the top of these things. I end disagreement over some very basic stuff. I was expecting to go to this conference, get all these clear, definitive answers and be better equipped to navigate 2024. And I am left with more questions, some more concerns. It’s wild, it’s absolutely wild. What’s going on? And I mean there were talks about LLM coercion between agents and how they can become manipulative, and there were examples of this. And then there’s also all this content around how LLMs aren’t good at generalizing information. So it’s all over the place.
21:26
But my predictions revolve around, unfortunately, how industry trends are currently going. So what I see happening is the large tech companies that are in power of the largest LLMs. It’s in their best interest to prioritize revenue. Unfortunately, this back and forth competition. We can expect to see a lot more generative AI baked into the most common tools that we use and love. So I fully expect to see those integrations soon.
22:02
Granted, it’s going to be with the large experimental warning to basically offload accountability for when these things go wrong, but I think we’re going to see a massive rollout into, possibly, applications where it doesn’t necessarily make the most sense, but they’re just trying to see what sticks. I also see there’s a really exciting area of startups that are focused on industry-specific LLMs and those narrow applications. So with increased performance for, say, marketing stuff, marketing tactics and strategy, those could do really, really, really well and we might start to see more and more of those in some of the most highly regulated spaces. So think finance, think medical law is another one. There’s a lot of applications within those spaces to save time and resources and also get more accurate in certain cases with very fine-tuned domain-specific LLMs that are connected to a reg-like model. They’re basically connected to a database of accurate, relevant information.
Danny Gavin 23:21
So I just want to ask a couple of questions. I want you to say that. So it’s interesting that you said that at the top there’s some disagreements. Do you mind sharing just one or two, Because that sounds really interesting.
Britney Muller 23:31
Oh my god, it’s crazy. It’s actually crazy. So let’s go very, very top. Yon Lacoon doesn’t agree with the godfather of AI and I think Yoshua Benjiro is on there as well but there’s general disagreements about risk, ai risk. So Yon basically stated there’s interviews I’ve watched several on YouTube where he publicly disagrees with these colleagues because he very plainly puts it that we have the control to connect these models to actions. So saying that these are just going to get out of control and how all of this high risk is ridiculous. We have the control to be like no, we’re not going to connect that to an action. That is incredibly foolish. Why would we do that? And because of that very simple kind of thought process you know he is of the camp of this is all being overhyped and it’s being overhyped for power gathering reasons, right Like, if you’re making it sound like, oh, we have god, knowing technology, we’re all powerful, people are going to be afraid and also, more you know, apt to give you money or invest in you or go to you as a resource. It’s BS, right? So, yeah, there’s so much of that right now.
25:02
Unfortunately, there’s so much hype and disillusion in the industry as well as a really terrifying segregation of concentrated power at the top, and that was so visible at NURBS, Danny, I can’t even put into words how frightening that was. So there are lots of incredible minority groups of researchers working on this technology and there was a dedicated day, a dedicated space for these researchers to talk about their work, to express what it is that they’re seeing and their concerns. And all these sessions they’re called affinity sessions are going on. They’re tucked back away on the second floor in these tiny rooms. No one attended them and it was at the same time these massive large language model talks were going on with some of the leading researchers and leading labs. So what do you expect? I push that back on NERP’s that you can’t create space for these affinity groups and not dedicate actual time and resources to encouraging people to listen to them. So that was really really hard to see and I’ve talked to several organizers about it that have confirmed some of these concerns. Yeah, so I really hope to help make a push on that. It’s actually fun fact. You could share this if you want the audience.
26:40
I was so heartbroken by the Indigenous NAI sessions. No one was there and these speakers were so proud, regardless of no one really being there, to just be there and be presenting at NERP’s. And Black in AI, muslim in AI, latina, I mean, there are all these incredible groups of researchers that got no visibility I mean none. So I took a morning off and I just put together the story about just that lack of visibility. And these groups are talking about losing languages, losing identity of their culture. It is totally different than talking about how we get more GPUs. It’s just like, oh my God, are we focusing on the wrong things? Where are we all doing? That was wild and very scary.
Danny Gavin 27:32
Well, thank you for sharing that with us. I don’t think the average person doesn’t realize what’s going on. That’s crazy. And it’s funny because, coming from the SEO world, where I’m not once again, I’m not saying we’re perfect, but I think at the SEO world, digital marketing world has made large strides in inclusion and trying to make it a little bit more open to multiple ideas, so I can imagine for you where you’re in that world, but now also in this world of AI. It’s such a culture shock.
Britney Muller 28:05
Such a culture shock and also goes against everything we know scientifically. That’s the other part where I’d like to even take general racism out of the conversation, the fact that it’s been statistically proven that diverse data sets improve the performance of models, diverse perspectives at the table improve the performance of outcomes. How are we still having such segregated conversations about this technology? And it’s a huge loss to the community because so many of these brilliant voices just aren’t being heard.
Danny Gavin 28:41
So I was gonna ask this question later, but I think it’s a good question to ask now. With LLMs and the training material, which are written by humans and they inherently have human biases, do you believe there’s any way to eventually filter out the human error and prejudices that we’re seeing?
Britney Muller 28:57
No, because there will always be bias. But there’s ways to mitigate that bias and one of those ways is by introducing transparency into what those biases are. So even with our rough understanding of the C4 dataset, which is used in most LLM training, we understand that Wikipedia is super biased. But now that we know the specifics of it because the average editor at Wikipedia is, I think, 27 years old, over 87% are men, they are of higher education, single and no kids Because we know that we can better adjust, we can better kind of navigate some of those biases. It’s when these closed black box systems give us no insight into what they were trained on, even like the image generative models. We lack the understanding there of what all went into it. If we knew, we could have a much stronger understanding of those biases and find ways to navigate them.
30:07
With LLM specifically, there is a really exciting area of research to help mitigate errors and some of the bias with multi-agent deployment. So with that you’re actually deploying basically multiple instances of LLMs and the thought process being that the odds of them all making the same bias, judgment or error will be less likely. There’s still a whole lot of work to be done in that space, but that’s some really exciting progress. And then, oh my gosh, I wish I could remember her name. There was a brilliant talk at NERBs about identifying bias in image recognition models and generative image models, and it was some of the most brilliant research I’ve seen on the topic to date. So there is exciting work being done but again, unfortunately it’s not necessarily in companies’ best interests, but it’s an important area of research.
Danny Gavin 31:10
So apparently AI has a problem with giraffeing Identifying giraffes, whether aren’t any, because its trading material contains more giraffes than are truly present in the world. I don’t know if you saw that article. Are there any other fun errors that you’ve encountered like that?
Britney Muller 31:24
Oh, there’s so many weird ones. Yeah, early on with Dolly, I was working at the AI company Hugging Base. We would play around with that all the time and nine times out of 10, if you didn’t enter text and you just asked it to generate an image, it was obsessed with generating a woman in a sorry state almost every time. And I interviewed the developer of that first, Dolly Mini, and she had no idea. He had a couple of hypotheses around like, well, maybe there was just a large amount of women in saris in the training set that had no alt text or caption. And then, obviously, Danny, I’m thinking of SEOs. I’m like, oh my God, what have we fed these things? What have we done? So, yeah, there’s all sorts of weird, weird oddities like that Crazy. They’re foolish in ways we can’t predict.
Danny Gavin 32:19
And as you’re talking, I’m thinking and that’s the scary part, because we’re gonna have a generation of people kids now who look at this as if it’s being absolutely true.
Britney Muller 32:29
Right, yeah, it’s crazy. The scary part is how is that shaping worldviews?
Danny Gavin 32:35
Yeah, yeah, so onto another kind of interesting topic, but OpenAI is currently in litigation for copyright infringement by the New York Times. What’s your take on how this might turn out and the impact on the marketing world?
Britney Muller 32:46
It could have huge implications. So I have heard that there was a similar, smaller lawsuit by a company I’m blanking on what they were, but they were able to sue OpenAI for like the same copyright reasons and now they get royalties every time like they believe that their tax has been used and so they’ve worked out some deal. I don’t know what they haven’t been like. That’s not fully transparent, but it’s super interesting to consider that something like that could happen with the times. But there’s also the conversation of, like what is public domain? How does that work?
33:23
You really start to get into like really really sticky situations, but I don’t think that some of OpenAI’s like biggest legal arguments are gonna hold water. I mean, the fact that they’re like one of their arguments is oh well, it’s really hard. Like you have to be pretty sophisticated, prompt engineer to get it to generate verbatim New York Times articles Like that’s not how copyright works. You know, like I don’t know that the lawyers will care about that or you know, so I don’t know. It’s gonna be really, really interesting. I think it will likely set a precedent. So it’s something to definitely keep an eye on.
Danny Gavin 34:01
It makes me think that maybe it will turn into something like a Spotify right. It’s kind of like if that piece of text was touched. I mean, I don’t know enough about LLMs to understand how that would work, but I mean, could it technically say okay, these data sources were pinged and I got to pay.
Britney Muller 34:18
I don’t know, yeah, but the thing, though, is it doesn’t even work like that.
Danny Gavin 34:22
It doesn’t work that way, right.
Britney Muller 34:23
No, it’s in a multi-dimensional latent space. It has no track record of like the sources of information. It’s just basically contextualizing all the text in like a language representation space of information. So it’s all mixed together. I don’t know how they would even do that.
Danny Gavin 34:44
It’s complex. I’m glad we’re not the lawyers or the judges in that case. Let’s segue into more of a SEO feel. What do you believe makes content world-class?
Britney Muller 34:55
Differentiating yourself from the status quo. I think that’s what makes you world-class. What makes you world-class is consuming all of the top-ranked content on a particular topic that you want to target and really identifying themes, topics, trends, layouts, media types. How are all these competitors delivering that information? And what’s missing? Not only what can you add, but how can you make it more succinct, how can you make it more value-driven for the user? What does it look like that their intent is? Are there a bunch of videos on the search result pages for these topics? Should you integrate that? Are images helpful in conveying different complex things? So I think being really, really mindful of how to deliver maximum value whether that be educating or getting people information fast, and doing it in a way that sets yourself apart through that value add.
Danny Gavin 36:00
And do you feel like Google really sees that Like you’ve done your research and you really feel like you have something that’s totally different and it deserves to be position one In your experience? Do you feel like more often than not, you can win with world-class content, or it just depends?
Britney Muller 36:17
No, that’s a good one, and I feel like I have a controversial answer and I just don’t care. I don’t even care if Google sees that it’s really not about Google at that point. It’s about maximizing your odds of being the most user-pleasing experience and having that sort of be the benchmark that Google will eventually see and likely reward. I think I focus a lot on that and I also, over the last I would say, four years or so, I have been putting way more emphasis on the old-school stuff that Eric Ward used to talk about.
36:59
He used to say you need to be link building. As if Google’s going to disappear tomorrow, where would your top visitors come from? And I love that. I think we should all always be thinking about that. So I will be on the most popular subreddits for weeks, if not months, ahead of a launch, knowing what they’re talking about, knowing what’s getting a bunch of interaction, and I will position different pieces of content into those communities in a way that’s enticing, and there’s opportunities to do that all over the web and I think that can also jumpstart world-class content. So, again, kind of bypassing Google is like Google’s going to start to see some of that traction.
Danny Gavin 37:42
I think that’s so powerful. We often say you need to create content for the users, not for the bots right, but it’s hard to actually do that and to actually think that way, and that’s why I think things like UX and UI are so important, and I don’t think everyone’s there yet right. But I love that perspective.
Britney Muller 38:01
It naturally brings that technical search value innately in the links and the resources. But yeah, I hear you.
Danny Gavin 38:11
So often it feels like the world is demanding quantity over quality as it pertains to content. How do you still satisfy consumers and algorithms with high quality content in lesser amounts?
Britney Muller 38:21
This is just speaking from my experience, my best kind of back pocket case studies that I have ready to go at any given moment to a potential client or a friend in a particular situation or someone hoping to accomplish this thing. They all come from a kind of rich, high time investment strategy. All of my prized gems in the search space and in marketing have really that have stood the test of time, or because I was able to really deep dive on a particular topic and deliver more qualitative content than quantitative every time. And it’s interesting because I was recently talking to a bunch of DMOs, so destination marketers, who are pressured to do this volume over quality, and I was able to give some very real examples of these things wouldn’t have stood the test of time if we took that approach. Granted, it doesn’t work for everyone, but I still get calls from clients from over eight, nine years ago who I set up on this piece of content that has snowballed over time. Those evergreen pieces aren’t going to be your low quality, high quantity work.
Danny Gavin 39:46
Yeah, I think the trick is, for how do you sell that to clients? Because sometimes I know myself. We were up against another company recently on a proposal for a specific SEO and from our perspective it was more about quality. But the other firm they were up against was all about quantity and they wanted to go with that other firm because of the quantity aspect. So it’s interesting.
Britney Muller 40:10
It’s so interesting and maybe some people can find a balance and do that well, but I also think of it as busy-ness isn’t a good metric of success and being really strategic in your efforts and also just seeing so many efforts get such a high splash because of the strategic nature of it. I think of the yard that created the celebrity jet emission research that went completely viral. I mean, their legal team was under fire because of it. They kind of stood their ground. It’s sit with their values. They’re very environmentally conscious, they want to work with companies that care about the environment, and it ended up working really really well in their favor, and now they have every backlink you could ever dream of, which is again just like a residual benefit. But I think of cases and I think as marketers we have gravely undervalued PR. There are so many ways we can be savvy with planting different stories in the press if we know what we’re doing, and so that’s also kind of been a tool I’ve enjoyed using a lot lately.
Danny Gavin 41:24
So we’re going to segue into a topic which I don’t know if you’re ready for, but I know that you are a certified ASL interpreter and back in the day, I believe you used to do it on the ski slopes.
Britney Muller 41:35
Yes.
Danny Gavin 41:35
You were helping people out. So for those people who don’t know what ASL is, that’s American Sign Language. It was developed independently of spoken American English and it’s the primary sign language in the United States. It has its own grammar and syntax rules. People have different accents in the way they sign. There’s also C S-E-E, which is signed exact English, exactly what it sounds like. Do you think?
41:54
so it’s just word for word signs to match up with spoken English. And then, naturally, you have your certified ASL interpreter, who’s someone who has taken and passed the certification exam, which is not easy. So, brittany, the question is how did you get into this? And I think it’s so wonderful and amazing.
Britney Muller 42:10
Oh my gosh, that was such a great intro to this. That is amazing that you did all that research. I love it. I yeah. Sign language is so fun, it’s so beautiful and I would argue you can be more descriptive in what you’re communicating, because you can play with how you’re communicating it. I can’t keep it real with you, danny.
42:31
I had a crush on a boy when I was in, let’s say, the sixth or seventh grade who was deaf, and so I convinced his mom to put me in sign language class with his brother, his younger brother, who I was of the same grade. So Jeff and I would take sign language class after school and Jeff had no idea. I would just have wanted to be able to talk to his brother, but it was very quick. Through Jeff I learned how fun it was, like we would be able to. He would be in a classroom and I would be in the hallway and we could talk. We could talk through walls, we could like. I remember we had swim class together and we could talk under water. It was like this is the most beautiful language ever. I was obsessed. I took it throughout college. Whenever I travel, I try to connect with different deaf communities. There’s a really beautiful community in Sydney, Australia, and they have a whole different language, just like the alphabet system that they use. Yeah, I just love it. It’s so fun.
Danny Gavin 43:35
So it was natural, and a position opened up on the slopes.
Britney Muller 43:39
That was another thing I kind of invented. To be honest, I needed a ski ticket, I needed a lift.
Danny Gavin 43:44
Tell me about that.
Britney Muller 43:45
I needed a little lift fast Now. I didn’t want to spend you know what $700 at the time. And so, yeah, I was like, do you have a? You know any ASL interpreters? Like, is that something that you need? And the organizer is like, actually, yeah, we could, we could definitely use that. And then I volunteered for the wounded warrior projects. That was really, really fun. I love doing that. The one thing I will say about interpreting on the mountain is your hands freeze, because there aren’t. I’m usually wearing a dark jacket and dark gloves and so I have to take my gloves off to be really specific in how I’m, what I’m saying or whatever, and so by the time we would go inside, my hands would just be frozen solid. They had to come up with better gloves now. But yeah, it was. It’s such a beautiful, beautiful language and it was a great experience.
Danny Gavin 44:39
So were you the ski instructor as well? Or you would like to stand with the ski instructor and then just interpret what he was saying?
Britney Muller 44:45
I was both on a snowboard. That was weird, because I can ski, but when it’s like those little hills like that, I am, even though snowboards kind of suck to be on. When you’re on like the bunny hills, I was just more comfortable on my board and so they. I was like I had to also negotiate that, but they let me keep my snowboard and then I would usually get teamed up with a snowboarder, but sometimes there were skiers and yeah, I would. I would teach people how to snowboard and ski.
Danny Gavin 45:13
Wow, Britney, you are a fascinating person, but knowing all these things about you now just takes that fascination and multiplies it by like a million. But it’s so cool, it’s just. But it all fits right. It all fits like the language and being that self, you know, creating your own situations and opportunities and having these amazing opportunities. Man it’s. This is a cool story. I love it and it’s only begun. Right, we got it. We got a lot more to write.
Britney Muller 45:43
Thanks, Danny.
Danny Gavin 45:45
So cool. So, to wrap up, we’re going to do our top three, because we were talking about skiing.
Britney Muller 45:50
Yeah.
Danny Gavin 45:50
Why don’t we talk about your top three places that you’d like to ski or snowboard?
Britney Muller 45:54
There’s so many that I hold dear to my heart and feelings. Most of them are out in Colorado. I think number one has to be Breckenridge, just because, like having lived there, you know we’re all like the secret little huts are and you know we’re like our friends would hide cases of beer and like snacks and you just knew where to go in the woods. So those like little forts and tree houses are so fun and there’s a bunch of like kind of more secluded, secret chairlifts that most visitors don’t know how to get to. So those little things are so fun. I also think A-basin back bowl is underrated. It’s so much fun on a powder day, Vail back bowl, you can’t beat it. And then they also like you know at your own risk, the mintern mile, where you go under the ropes and it’s a mile long into the town of Mintern behind Vail. But it’s if you go with a group of people who know what they’re doing. It’s so fun, it’s so beautiful. So, yeah, those are probably my top three.
Danny Gavin 46:56
This is going to be embarrassing but I have to say it because you bring up the Vail back bowls. So it was my first time skiing. I took a couple lessons at the bunny hills at Vail. I was with a whole bunch of brother-in-laws and their whole crew. They all knew how to ski and I was like the newbie. So I took the lessons for one day, did pretty well and they saw me like Danny, you’re ready, ready to go. So they took me to the top, to the back bowls and you know, once you get down it’s cool, but when you first start that incline is just hectic and literally it was the worst experience of my life.
47:31
So I like to go down a little bit, but I’m so scared and like no, it’s easy, this and that, and I literally like I ski down, like sitting on my skis, going down the back, oh my God. Eventually, I think ski patrol saved me, but I’ve skied many times since but that was my introduction and that’s a rough introduction, Danny.
Britney Muller 47:55
I promise it can be more fun. That’s brutal.
Danny Gavin 47:59
But it’s funny. It’s funny to look back.
Britney Muller 48:01
So it’s all good.
Danny Gavin 48:03
So, before we wrap up, I wanted to know what’s your next big thing, what’s the big plan for 2024?
Britney Muller 48:08
The mission for 2024 is to basically get this information into the hands of marketers and SEOs in a way that is immediately valuable for them. I hear a lot of conversations about, you know, marketers being put in the position to use AI by upper management, that they themselves don’t know what this technology is. So I have different resources in the pipeline just to help with those conversations, help navigate this changing landscape and also how to take, like more actionable steps to incorporating this into your workflows. There are a lot of incredible applications, you know. Customer support comes to mind. There are some really great examples by OpenAI and I’m excited to kind of repackage and like tailor things for marketers. That can be really helpful.
Danny Gavin 48:59
So is it possible there might be a course coming down the line.
Britney Muller 49:03
Possibly.
Danny Gavin 49:04
Okay, you heard a first tier, hopefully, so where can listeners learn more about you and your business?
Britney Muller 49:09
I’ve really been putting blood, sweat and tears into the large language model guide on data, side 101. Parts one and two are up. Three are drafted and almost ready to go, so that should be coming very soon. But I want that to be kind of a standalone resource for people to reference back and get more information. I also, as my dear friend Daisy Quaker always puts it, she’s like you know, we as marketers we bake these turkeys, right. We spend all of our time putting together these huge guides or resources and we get these turkeys.
49:43
You have to make turkey sandwiches, you know like, turn it into, like videos and I don’t know just different resource types, right, Like audio think of all the ways that people can consume content. So I want to be more mindful of doing some of that this next year and delivering it in a way that, again like, provides value and helps communicate this technology and make it accessible. That’s the other thing. Like even attending NERP’s, there’s a lot of gatekeeping behind this technology. It stands behind a lot of closed doors, but there is a ton of information available to make sense of some of the things that we’re interacting with. So I really want to kind of tackle some of that in 2024.
Danny Gavin 50:27
So cool. Well, Britney, thank you for being a guest on the Digital Marketing Mentor and thank you, listeners, for tuning into the Digital Marketing Mentor. We’ll speak with you next time.