But first, the complaining.
As I have personally discovered, having a slow internet connection will make you painfully aware of how bloated many modern websites are. A lot of major websites out there need to download megabytes of JavaScript just in order to be functional. Obviously, a lot of web apps rely heavily on JavaScript for their functionality, and this is basically unavoidable. But a lot of websites really don’t. As a quick test, I reloaded a Quora page I had open for research I did for this diatribe and used the network monitor to check how much JavaScript it downloaded. The result was 46 separate files totaling around 6.4 megabytes. That’s after adblock and Firefox’s tracking prevention. Far from the worst I’ve seen, but this is for a website that’s based around what is mostly static text. For reference, the actual content of the article you’re reading right now is about 40 kB, or 0.04 megabytes.
And it’s not just data transfer that’s the problem with these sites, a lot of those unnecessary scripts run constantly, polling for interactions, phoning home with analytics data, and so on. Firefox is currently using over 6 gigabytes of memory on my computer, and around 20 percent of my CPU. I’m not interacting with it, it’s not playing any media, it’s not even visible on my monitors right now. This is ridiculous. And of course this doesn’t even touch on how annoying these scripts can be to deal with. Pointless JavaScript often breaks totally basic functions of a website, like, clicking on links. Clicking on links! I’m sure anyone reading this has at some point tried to right-click or ctrl-click or middle-click a link, any of which should allow you to open the link in a new tab, and have it do exactly nothing, because it’s not a link, it’s a plain <div>
element with a JavaScript event handler on it listening for click events so it can redirect you to another page. Browsers already have links! Please stop replacing them with your own worse ones!
And here’s another thing. It’s an important one, but maybe not one you’ve ever thought about before. How hard do you think it would be to write your own web browser? Not a fully-featured, fully-compliant browser with all the bells and whistles, just being able to load and render an average webpage. The answer is way more work than one person could ever be expected to do. The endless features and anti-features that keep getting tacked onto browsers and added to web standards make web browsers exceptionally complicated programs. And at this point, about 92% of users use one of three browsers[1]: Chrome, Firefox, or Safari, or Edge which now uses the Chromium engine under the hood so it’s essentially just Chrome with a different paint job. And for something as indispensable to the average person as a web browser, something most of us use for important tasks every single day, don’t you think that lack of variety is troublesome?
I mentioned tacked-on features and I want to come back to that. By my count, CSS, just counting the bits that are at least a working draft and therefore might be implemented in browsers, there are 37 data types, 60 pseudo-classes, 15 pseudo-elements, 35 at-rules, 150 functions, and a whopping 549 properties! And keep in mind that most of these are in various different states of implementation in different browsers, sometimes with unique quirks in specific browser implementations. And all of this is just for styling web pages. I’ve used a lot of modern CSS features, and using CSS nowadays is definitely nicer than it used to be. But do we really need over 500 unique properties? Again, think about how that limits implementations of compliant CSS parsers and renderers basically to professional development teams - it’s so far beyond the scope of what a hobbyist can do. And since so many websites rely on this massive pile of features, any web browser that only implements a modest subset will be unable to correctly render a lot of websites. And JavaScript is a mess I could write another whole post about, but suffice it to say it’s gotten quite bloated and funky too.
It’s not just client-side JS and CSS, the HTTP protocol itself has had features tacked on to it over the years as well. The complexity of the web has provided plenty of tools for tracking users and collecting information on them. For example, a JavaScript API named “evercookie” is capable of storing persistent information in a user’s browser using no less than sixteen different methods, including regular cookies, abusing resource caching, storage space provided by Flash, Java, and Silverlight, and even storing data in browsing history[2]. All of these methods store persistent data that can be read back later, and evercookie automatically recreates all of the ones you try to delete if it detects even one remaining data source, making it extremely difficult to fully remove. And there are other methods of violating user privacy, such as, obviously, services collecting information linked to logged-in users, and less-obviously, browser “fingerprinting”. This is where a site abuses the fact that web standards are such an inconsistently-implemented mess in order to identify - sometimes down to the specific version - what browser you are using, by testing for the particular set of features your browser implements, which in some cases can make it possible to link activity back to you. And since this kind of fingerprinting is capability-based, even if you take measures to avoid identifying your browser, such as spoofing[3] your user agent[4], sites can still perform this fingerprinting to leak information about your browser.
Ads are a common annoyance on the web, to the point that many users (such as myself) use browser extensions specifically designed to prevent any and all ads from loading, at least to the extent that is reasonably possible. This practice receives a lot of criticism, saying that blocking ads denies web publishers access to ad revenue. And yes, this is certainly true. But it misses the larger picture about ads, which is that they’re not just annoying, they’re security risks. Ads have always presented a number of security issues. Many advertising platforms are minimally filtered, if at all, so malicious actors can easily add malicious ads to a service and have them served on hundreds or thousands of websites. Even if the ad platform itself filters submitted ads sufficiently, if the platform is compromised then hackers could inject their own malicious ads to be served across the whole platform.
Many of these ad systems will send a user through several intermediate links after clicking on an ad, before reaching the destination website. These intermediate sites are used for tracking and analytics purposes. If any one of those intermediate sites are compromised, that’s another attack vector. Occasionally browser exploits are found that use specially constructed media to compromise a browser, such as a specially-designed image that causes a buffer overflow in a vulnerable browser renderer. These exploits are usually found and patched fairly quickly, but despite developers’ best efforts, they do keep popping up from time to time, and such exploits make any image or video being pulled from an unknown, untrusted third party a potential security risk. And perhaps the most dangerous kind of “malvertising” is the simplest of all - tricking users via ad design. Frequently, malicious advertisers will create ads that are designed to look legitimate, like perhaps a download button, appearing conveniently in a banner ad right before the actual download button. These are especially devious since sometimes you will be trying to download an executable, like perhaps a program installer. So the malicious ad redirects you to a site that serves you a “setup.exe”. If you missed any subtle clues that this link was illegitimate, you better hope your antivirus catches that file before you run it.
These are threats that can affect even tech-savvy users, especially the media exploits. And since these days everyone is an internet user, a huge portion of regular internet users are non-tech-savvy folks who are much more vulnerable to malicious ads. And yet, even after all of this, sites that use ads for revenue will still put insulting messages behind ad elements to shame you for using an ad blocker, even as they try to push untrusted content onto your computer.
But it’s not just security that’s problematic about advertising. A distressing amount of the web is designed around advertising. Companies redesigning their sites to maximize ad exposure, using analytics data to optimize ad clicks, performing algorithmic filtering of content to try to serve you more, or more strongly personalized, ads. If you take a look at Twitter’s quarterly results[5] you’ll find that the primary statistic Twitter uses to measure user activity isn’t total users, or active users, it’s monetizable daily active usage. Twitter measures their success or failure based on how much monetization they are able to perform. And from a business standpoint, that makes sense. They’re a business, and they’re trying to make money. But from a personal standpoint, what does this mean? It means that if you use an ad blocker, or you post explicit content, or you tweet with blacklisted words that disqualify you from certain advertising, Twitter does not care about you or your experience on Twitter. The only users that matter are the ones that are marketable. And basically all other ad-focused, for-profit platforms are going to follow that same methodology. If you are someone who actively tries to protect your privacy and security by blocking tracking methods and advertisements, you are persona non grata to these platforms.
Accessibility is an important but often-overlooked aspect of human-computer interaction. It’s easy to forget that there are people with poor eyesight, or poor color vision, or who are blind, or deaf, or have a physical disability, or have any other accessibility need, when you’re perfectly-abled yourself. Modern HTML has some built-in accessibility features, like the use of <nav>
to point out navigation links, and the use of alt
attributes to describe images for users with screen readers. But accessible design goes much further than that, and many web devs have some bad habits that make websites less usable for people with disabilities. Things like using elements for the wrong thing, like the link that’s actually a scripted <div>
I complained about above, or using tables (which are supposed to display data in tabulated form) to lay out pages, or abusing HTML’s leniency to write poorly-structured text that screen readers struggle with. The primary issue here is that HTML was designed in the ’90s, and accessibility simply was not a concern when it was being developed. No one could have predicted that the web would become as ubiquitous as it is today, and so all of the accessibility features we have now are just awkwardly tacked onto a markup language that’s barely changed in 30 years to maintain backwards-compatibility. Now the responsibility to ensure websites are accessible to users with disabilities falls squarely on web designers, and time has shown that most of them won’t even bother.
A different - but still very important - type of accessibility issue is the difficulty of creating content on the web. The mess of HTML, JS, and CSS, all of which are horrible incrementally-updated backwards-compatible nightmares, makes it harder than it needs to be for new users to create content online. Modern tools do exist for beginner-friendly web content creation, with varying levels of usefulness and bloat, but in the end they still need to transform their output into the same formats described above, sometimes introducing more issues in the process. The success of Flash content online can at least partially be attributed to the fact that it was fairly easy for non-tech savvy folks to create complex interactive content that could be published online and appear and function exactly like they designed it. Since then, Flash has become a whole different mess on its own, but it’s a useful lesson in usability.
And speaking of problems with web publishing, it’s even more difficult to publish content online in a way that works correctly and is discoverable by users, unless you go with one of the website-designer… websites, that let you publish content on a subdomain or sometimes even with custom domain support. But most of these services are paid, or the free version is horribly limited, or they’re full of ads, or they work poorly and make designing sites miserable, or some combination of these problems. And ultimately they all limit the user’s creative power by only letting them use whatever point-and-click functionality they happen to implement, whatever premade widgets they happen to provide, whatever arbitrary limitations they put on the design. And of course there’s no interoperability between these services, so if you want to switch to a different service you have to start over your design from scratch. Like most of the things I’m complaining about in this post, these things have gotten better over time, but it’s still indicative of some serious underlying accessibility issues with publishing on the web. For something so fundamental to our daily lives, and communications with others, and content creation, do we still need to be using this hulking 30-year old amalgamation of a platform? Wouldn’t it be nice to have a platform that lowered the barrier to content creation and publishing? (and yes, I know there are some, I’m addressing that in a bit)
The most obvious case of centralization on the web is in the form of the most popular web services. Want to have control over your own emails? Too bad. The centralization of email services means that it’s functionally impossible to reliably self-host email in 2022[6]. Want to host easily-accessible video content that’s discoverable by users but don’t like YouTube? Too bad. No other video hosting platforms come close to YouTube’s viewership or content availability. YouTube’s search is the third-most-used search engine in the world, behind Google and Google Images[7]. Want a simple microblogging service like Twitter but don’t want to use Twitter? Well actually, Mastodon has been doing pretty well in that regard[8], and it’s a big inspiration for my own work. But it still doesn’t come close to Twitter[9]. And as many users of social media or other web apps are already aware, many of these centralized services offer little to no interoperability amongst themselves. Being competitors in a capitalist economy, these sites are actively discouraged from letting their users share their data with other apps. I was going to add another aside acknowledging the recent collapse of Twitter, but actually this highlights another problem with centralization, which is that when the monolith falls, the small competitors previously starved of use now need to race to try to meet that same multi-billion dollar quality of service (and there’s a lot of catching up to do) so users often feel stranded with nowhere to go.
Another problem with centralized services - one any Twitter or YouTube user should be familiar with - is that they’re impossible to moderate effectively. Massive platforms like YouTube and Twitter with users in the hundreds of millions to billions have very few actual humans performing moderation, and have instead largely turned to automated systems to do their moderation. The moderation scene has gotten bad enough that on YouTube, videos are being copyright claimed for using white noise or cricket sounds[10], and on Twitter, accounts are being banned due to tweets with just a single flagged word in them (in one case, the word was “Memphis”[11]).
There are also serious security implications posed by the centralization of internet infrastructure. DNS servers - the servers that resolve domains like google.com
into an IP address like 142.251.33.78
so your computer knows what address to connect to - are becoming increasingly centralized as of late, where in some places like New Zealand and the Netherlands, around one-third of all DNS traffic goes through just a handful of DNS providers[12]. Why is this a problem? Well, the more centralized DNS services become, the more they become targets for both criminals and government surveillance. In 2016, a denial-of-service attack on just one DNS provider, Dyn, brought down several major sites, including Twitter, Netflix, and Reddit, for most of a day[13]. Attacks like this will only become more serious as fewer and fewer services provide the majority of DNS traffic. And the threats posed by centralization extend beyond being unable to watch online movies for a day - in 2017, an exploit in CloudFlare’s content delivery services, which at the time was in use by over 5.5 million websites[14], exposed sensitive user information that was supposed to be protected by TLS, including “full https requests, client IP addresses, full responses, cookies, passwords, keys, data, everything”[15]. According to a report from CloudFlare, this leaking of data likely happened more than 18 million times before the exploit was fixed, with sensitive information also being cached by search engines[16]. This even included the two-factor authentication app Authy, meaning even accounts protected by 2FA could have been compromised[17].
Perhaps the most obvious issue with identity on the modern web is how often it is tightly coupled to one’s real-life identity. How many sites or services have you signed up for that asked for your first and last name, even though there was absolutely no need for it? Many websites also require a mandatory valid phone number, which is even more unavoidably coupled to a real-life identity since any valid phone number is registered to one real-life person, and this information can be obtained from the phone number alone. These requirements also make it difficult if not impossible to maintain more than one identity online (unless you’re willing to pay for multiple phone plans).
Another well-known problem with web-based authentication is how it requires you to let a third party manage your security. The main reason why security breaches have been such a high-profile problem lately is because modern web systems require you to store your information - often information that can be used to identify you - on someone else’s web servers, and just hope that they have competent security. Well, that hasn’t been going so well, has it? This is also the reason why you need a separate password for every account you use: authenticating users with a username and password, and then storing those usernames and passwords in association with that service, means that a security breach can potentially give the attacker everything they need to access and control your account, and if you’ve used the same username and password elsewhere, those accounts too!
This is also the reason why you need to make new accounts everywhere you go (and use separate passwords for all of them). The way authentication is currently done on the web necessitates this kind of needless inconvenience and security risk. And single-sign on (SSO) services don’t fix the problem either. Like to log in everywhere with your Gmail or Facebook account? Well now, if your Gmail or Facebook account is compromised, everywhere else that you’ve used that account for SSO is also compromised. We’re back to square one. And of course, Gmail, Facebook, and similar services demand your full name and phone number to create an account, so those are linked as well, making these SSO credentials an even more attractive target for hackers.
Another issue with modern web identity is that almost no services expose facilities for cryptographic verification. Got an email that appears to be from a family member but was actually a phishing attempt? Seen accounts impersonate someone by using a username that looks the same, maybe with an I instead of an l? Worried (justifiably) that someone might be intercepting, blocking, or altering messages to or from you? All of these problems would be solved (at least mostly) if services provided the ability to easily cryptographically verify the author of the content.
So, as someone who isn’t satisfied with the current state of the web, what am I to do? Are there answers to these questions? Solutions to these problems? Find out next time on–
Ahem. The answer is yes, there are solutions, but they aren’t enough. I’m going to use two examples of systems that take steps in the right direction - Mastodon and Gemini - and highlight what they did right and what they still have problems with.
Mastodon claims to be an alternative to social media sites like Twitter or Tumblr, one that is decentralized and decommercialized. And it does solve a lot of the problems I talked about above. It is decentralized - made up of thousands of individual “instances” that all communicate with each other using a shared system to form a large, (mostly-)contiguous network. And the “mostly” qualifier there is a good thing; it’s not fully contiguous because instance administrators can choose which other instances they want to “federate” with, that is, exchange content with. This is a powerful tool to completely cut off harmful communities, in a way that each instance can control. Mastodon also encourages building communities in healthy ways (instead of monetizable ways) and providing better privacy and moderation. That last point is especially true; moderation on Mastodon (on some instances, anyway) is orders of magnitude better than the automated keyword police on Twitter. Why? Because since the individual instances are small, they can be reasonably moderated by a few people. Decentralization solves the moderation scaling problem.
Mostly. One problem highlighted by the recent Twitter exodus is that Mastodon is only partially decentralized. Some instances have become quite large - the current largest, mastodon.social, has over 832,000 users at the time of writing - and when instances become so large, they encounter many of the same problems that completely-centralized services have, and struggle to keep up with moderation as they scale. And with so many new users fleeing from Twitter, some servers have had technical issues with scaling as well, sometimes resulting in downtime and multi-hour delays with content federation[18].
Also, Mastodon by design cannot address the issues with the underlying web technology - HTTP, HTML/CSS/JavaScript - because it’s still built atop the same stuff. However, there are other systems out there that attempt to tackle this problem in their own way, such as…
Gemini is a lightweight an application-level protocol (and alternative to HTTP) that focuses on minimalism, privacy, and self-publishing. It solves a lot of the problems with the modern web by simply… not implementing them. By freeing themselves from the burden of compatibility with HTTP, they no longer need to include features such as cookies and caching that can be used to track users and collect data on them. This makes browsing Gemini sites a very safe and comfortable experience - you know you won’t be tracked or advertised to, because the platform literally does not support it. Gemini is also designed with one kind of accessibility I mentioned earlier in mind, that is, ease of implementation and publishing. Gemini is designed so that someone could write a reasonably usable client application for it - a web browser, more or less - in around 200 lines of code. Compare that to the roughly 35 million lines of code in the Chromium browser engine[19].
Gemini’s lightweight implementation also means that Gemini sites can load much, much faster on slow internet connections, since the total amount of data needed to display a page is in the kilobytes instead of megabytes. The lack of heavy scripting and styling also means that Gemini clients run much more smoothly on low-end systems with limited computing resources. The decision not to include an equivalent to CSS for styling was also made in favor of user freedom: without a strict styling regimen dictated by the target website, clients are free to style pages however they want. No more dealing with sites that don’t have a dark theme - when your browser chooses the styling, everything can have a dark theme.
But Gemini is also very limited compared to the Internet. The same lack of styling that promotes user freedom for visitors also stifles creative expression for publishers by making them unable to control the styling of their own content. The lack of scripting means that the functionality of Gemini sites is severely limited. These points aren’t such big issues when you consider that Gemini is primarily designed for serving plain text content - blog posts, stories, poetry, and so on. But it does significantly limit the scope of what Gemini can be used for.
Mastodon and Gemini are just two examples of projects that attempt to solve some of the issues we face using the modern web. There are many more out there, and I do hope that they gain more traction soon and spread awareness of the fact that we don’t need to be stuck with the One True Internet, or One True Social Media Platform. But none of the options available today (at least that I have found) effectively address all of the problems I am concerned with.
And that is why I am proud to announce that I am adding yet another obscure alternative to the list, to make things even more confusing!
In all seriousness, I am currently developing an entire network technology stack - communication protocols, networking systems, and yes, social media - that hopes to go even further beyond what options currently exist. Fully open-source, fully decentralized, fully decommercialized. In particular, there are a few pitfalls I want to avoid with this project which I will outline below.
There is a problem with nearly all social media systems, which is that no one will want to use it until there are people that use it. This is a nasty catch-22 that must be overcome in order to get any social media platform off the ground, where users need to be given good reason to switch over to a different platform, and generally that means their friends need to also be on it. The way I hope to overcome this is via a unified framework for content publishing and social interaction that integrates as fully as possible with existing platforms. “Come to Not-Twitter! There’s no one here yet but I promise it will be cool!” becomes “Come to Not-Twitter! It’s better and it also has Twitter!” which is a much more compelling argument. By making a layer that integrates seamlessly with existing platforms and new platforms alike, the “social media problem” is avoided by allowing a smooth transition from one service - from one social network - to another, with all the shades of grey you want in between.
As any artist will tell you, no art hosting platform is perfect. DeviantArt? Tumblr? Twitter? ArtStation? All of these have significant problems, and moreover, different problems for different users. Many have tried - and failed - to create a perfect platform, but they will always fail because no one can create an environment that fits everyone’s uses perfectly. Instead, the users themselves should define the environment they want. Imagine a universal system where users can publish stories, post art, show off their music, write tiny shitposts, and do whatever else one might want to do online, and every publisher gets to choose how their content is displayed to new viewers, and every viewer gets to decide how to display that content along with all of the other creators in their own personal feed.
The basic idea is that multimedia content will exist independently of a specific interface, and that content can be displayed in any number of ways depending on user needs. When you visit someone else’s personal page, they can showcase their content in whatever way they like. Maybe that’s showing off their latest album, or displaying long-form writing, or arranging a video player with space below for comments, or showcasing their latest artwork while also clearly showing their commission availability and prices. Meanwhile, when users “follow” or “subscribe” (or whatever else you want to call it) other users, that content gets aggregated and presented in a way, again, fully customizable. Users could set up multiple feeds, one displaying all of the microposts from Twitter, Mastodon, or any other platform, one feed showing new videos from creators you’re subscribed to on YouTube, as well as videos independently published on decentralized systems like PeerTube, one feed displaying art from artists you follow, and so on. And each of these feeds can be customizable to display content in exactly the way the viewer wants, in whatever layout suits it best.
Sound impossible? Probably. But I’m dumb and stubborn enough to try anyway.
Peer-to-peer networks have plenty of unique strengths, and unique weaknesses. The biggest weakness for a social system is discovery: how do you make connections with new people amongst a giant, amorphous mesh of millions? For the solution to this problem, I take inspiration from Mastodon’s mostly-decentralized architecture. Although the network is fully decentralized and works perfectly without any centralized authority, there’s nothing preventing centralized “hubs” or “relays” existing for various topics, so that people can more easily find others with similar interests. If you like listening to heavy metal and talking about woodworking, you can check out a heavy-metal-focused relay to find new musicians to follow (or if you make music, publish your own work to that relay for others to find), and you can subscribe to, and publish to, a woodworking-focused relay to exchange Twitter-esque microposts and engage in live chats with folks about woodworking. And if those relays go down? Maybe the server owner can’t afford to keep it running? Or the apartment they host it from burns down? You still know about all of those people you found, and you can track them down through the P2P network to find them somewhere else. The network is distributed and fault-tolerant while still being user-friendly and promoting discovery.
And speaking of fault-tolerance, this network can also be asynchronous. This means that you could write some blog posts, reply to some comments, and do whatever else with your locally-cached content while offline, and whenever you connect to the network again, those actions will be synchronized throughout the network, you can fetch the latest new stuff, and then go offline again. This is especially important if you’re concerned with off-grid networking, evading surveillance, or just have an unreliable network connection, or travel a lot.
How to address the problem with web technology? JavaScript bloat? Fingerprinting and HTTP tracking methods? This is another case where jumping ship entirely wouldn’t work because you need users for a network to… work. Instead, the system will exist as a generic, protocol-agnostic abstraction layer which can communicate over many different protocols. You can host fancy web pages over HTTP, publish blog posts on Gemini, distribute your music over IPFS, send dank memes to your friends over ham radio. Similar to the unified, abstracted content presentation philosophy above, this system aims to be able to work with both existing and new network protocols, to provide as much flexibility and support as many use cases as possible, while still connecting users with different environments, disparate tastes, and unique use cases.
Fair warning, this one’s a bit complicated.
You may have noticed that Mastodon, despite decentralization being a strong focus, still uses centralized authentication. That is, you still sign up with account credentials that are stored in a web server somewhere, becoming a more and more enticing target for hackers as the platform grows, and you still need to remember passwords and worry about grabbing the username you use everywhere else. This was an intentional compromise made when designing Mastodon, since the alternative - fully-decentralized peer-to-peer networking - is much more complex, both in terms of implementation and use. However, decentralized identity doesn’t need to be complicated. The solution lies in the form of a “web of trust” model, which users of PGP will likely be familiar with.
The overall concept is that you generate a cryptographic key pair - one public and one private key - and use the public key itself to represent your “identity”. Your private key is used to facilitate encryption and verifiable communication, which can be verified as authentic using the public key you pass around, which also identifies you uniquely among the whole world. No usernames, no passwords. Instead, users can endorse someone as “being the real so-and-so” by signing the person’s public key with their own private key. This signature is cryptographically verifiable, so that if you trust A, and A trusts B, and B trusts C, you can be reasonably sure that C is who they claim to be, even if you’ve never met them and they’re not a part of your social circles. And also, since key pairs don’t have real-life information like names or phone numbers attached to them, they are also anonymous. Your online identity can be totally coherent and consistent across all spaces, while also being decoupled from your real-life legal identity. In fact, you can have multiple key pairs that represent multiple different identities, and use those identities however you want to achieve whatever level of privacy you desire.
A common argument against the web-of-trust model is one similar to the social media problem, namely that you need people in the network you trust in order to have people in the network you can trust. However, this can be alleviated in a few ways. One is through the use of variable levels of trust. You trust a close friend much more than a casual acquaintance, and so their endorsement means more. Someone you’ve known for a long time can be given a strong trust value, and therefore be a useful tool in your verification toolbelt, while other people can be trusted at a much weaker level, possibly as little as “I know that they exist and claim to be someone”. And then, over time, the people you get to know better can be assigned higher levels of trust to strengthen the web of trust you’re building.
Another way to solve this problem is by what I call “reasonable verification” through side-channels. The only way to be absolutely certain that a public key belongs to the person (or persona) they claim is to physically meet them in person and obtain their public key from them directly. This is the only way to be 100% sure that no one has tampered with anything along the way. But this doesn’t mean the system is unusable otherwise. Yes, someone could intercept a public key sent over an unencrypted channel and tamper with it, so you can’t be totally sure. But if you have a cryptographic signature, much shorter than the key itself, which can be used to verify the authenticity of the key itself, you can easily communicate that short signature over other channels. For example, maybe you email your friend a quarter of the signature, send them another quarter over Telegram, another quarter spoken over the phone, and the final quarter by snail mail, that way an attacker would need to compromise four different communication channels simultaneously in order to fudge something. Are any of those channels 100% secure? Is this method 100% airtight? No. Is it more than good enough for nearly all cases? Yeah.
More importantly though, how often do you actually need 100% certainty that someone is who they claim to be? Maybe for financial exchanges or formal contract agreements, sure, but for sharing a picture of your cat with internet strangers? Who cares? The web of trust model can be used to verify the a key’s authenticity when needed, but the system can work fine for most purposes without it. The strongest defense against impersonation is simply keeping a copy of someone’s public key associated with their name so you’ll know if someone with a different key claims to be them, and this can all be done in software without any user intervention or understanding of cryptography needed.
Everything I’ve mentioned up until now may sound a bit complicated. But don’t worry! In reality it’s much worse.
This massive omni-network project will necessarily contain a ton of complexity, and if blog posts complaining about not understanding Mastodon are any indication, that complexity can push users away before they get the chance to see the benefits of a system. This is why I’m going to put extra effort and attention towards making transitioning to this system from other platforms and networks as seamless and foolproof as possible. Coming from Twitter and just want a Twitter-like experience someplace else? Click the “just give me Twitter” button during setup and your interface, feeds, default networking settings, and so on will automatically be configured to mimic Twitter. And since the platform integrates with Twitter, you can pull in tweets from, and post tweets to, the existing Twitterverse with minimal hassle. Ideally switching to this system should be no more complicated than opening up TweetDeck, but with the added knobs under the hood that users can tweak to customize their experience as much (or as little) as they want.
There’s still plenty more to cover, and certainly things I’ve gotten wrong or forgotten to mention, but this post is already over 6400 words and several days in the making and I’d like to get this thing out the door, so I’m calling it here. If you want to hear me complain even more, you might have a problem. But it’s one you can indulge with my Mastodon account, where I will probably complain a lot more about things. If you’re interested in this project, let me know! I sure as hell ain’t gonna pull this off myself - I’m already working with Kasran (Twitter, Mastodon) - and help or discussion is always welcome. You can contact me at the Mastodon account above, or on Twitter at @trashbyte, or email me at hello@trashbyte.io. I’ll also be publishing updates here, but I’ll be most active on Mastodon.
To anyone who actually read all of this: thank you and/or I’m sorry. Really, your interest means a lot to me <3.
[1]: Browser Market Share Worldwide.
https://gs.statcounter.com/browser-market-share[2]: evercookie - virtually irrevokable persistent cookies.
https://samy.pl/evercookie/[3]: User Agent Spoofing: What Is It & Why Does It Matter?
https://www.clickcease.com/blog/what-is-user-agent-spoofing/[4]: User agent - MDN Web Docs Glossary: Definitions of Web-related terms.
https://developer.mozilla.org/en-US/docs/Glossary/User_agent[5]: Twitter Announces First Quarter 2022 Results
https://web.archive.org/web/20221015174950/https://s22.q4cdn.com/826641620/files/doc_financials/2022/q1/Final-Q1%e2%80%9922-earnings-release.pdf[6]: After self-hosting my email for twenty-three years I have thrown in the towel. The oligopoly has won.
https://cfenollosa.com/blog/after-self-hosting-my-email-for-twenty-three-years-i-have-thrown-in-the-towel-the-oligopoly-has-won.html[7]: YouTube 2nd Biggest Search Engine - The Myth That Just Won't Die.
https://www.tubics.com/blog/youtube-2nd-biggest-search-engine[8]: Mastodon - Fediverse.Party.
https://fediverse.party/en/mastodon/[9]: Most popular social networks worldwide as of January 2022, ranked by number of monthly active users.
https://www.statista.com/statistics/272014/global-social-networks-ranked-by-number-of-users/[10]: Twitch clip: Video claimed for cricket sounds.
https://clips.twitch.tv/embed?clip=OptimisticGleamingLardDoggo&parent=example.com[11]: Don't Tweet This! Twitter Automatically Blocks This Word.
https://www.techtimes.com/articles/258015/20210314/dont-tweet-this-twitter-automatically-blocked-this-word-learn-more.htm[12]: How centralized is DNS traffic becoming?
https://blog.apnic.net/2020/11/24/how-centralized-is-dns-traffic-becoming/[13]: DDoS attack that disrupted internet was largest of its kind in history, experts say.
https://www.theguardian.com/technology/2016/oct/26/ddos-attack-dyn-mirai-botnet[14]: Cloudbleed Explained: Protect Yourself From the Internet's New Security Flaw.
https://www.popularmechanics.com/technology/security/a25380/cloudbleed-explained/[15]: Issue 1139: cloudflare: Cloudflare Reverse Proxies are Dumping Uninitialized Memory.
https://bugs.chromium.org/p/project-zero/issues/detail?id=1139[16]: Change Your Passwords. Now.
https://gizmodo.com/cloudbleed-password-memory-leak-cloudflare-1792709635[17]: Incident report on memory leak caused by Cloudflare parser bug.
https://web.archive.org/web/20170223233000/https://blog.cloudflare.com/incident-report-on-memory-leak-caused-by-cloudflare-parser-bug/[18]: Scaling Mastodon in the Face of an Exodus
https://nora.codes/post/scaling-mastodon-in-the-face-of-an-exodus/[19]: The Chromium Open Source Project on Open Hub
https://www.openhub.net/p/chrome/analyses/latest/languages_summaryNote
I wrote this post a while ago, on the previous version of my site. I've included it here cause I think it's Pretty Alright.
Welcome to part one of a 3-part series of tutorials on making custom Blueprint nodes in UE4. In part one, I’ll walk you through the creation of a Blueprint Library in C++. If you don’t know much about programming, don’t worry, it’s pretty simple! In later parts I’ll dive into the nitty gritty stuff, starting with defining custom “thunk” functions in order to properly handle wildcard parameters.
Quite simply, a Blueprint Library is a C++ class with a bunch of functions in it, which use some special UFUNCTION
specifiers to automatically generate Blueprint nodes for each function. It only takes a few lines to create a custom node, so let’s jump right in.
To start, open up an existing project or make a new one for this tutorial. If you’re making a new project, make a “Basic Code” C++ project. If, after hitting “Create Project”, it opens Visual Studio and just sits there without doing anything, go to the project folder and open the project file yourself.
Go to “C++ Classes” in the content browser and open the folder for your project. Right click in the content browser and select “New C++ Class”.
Check the “Show All Classes” checkbox in the top-right corner, and find BlueprintFunctionLibrary
in the list (type in the search bar to filter the list).
Give it a name and hit “Create Class”.
After it compiles the new class, the files for it should appear in the Visual Studio project. If VS isn’t open, you can open it with File > Open Visual Studio in the editor, or by opening the solution file in the project folder. You should have two new files: <YourClass>.cpp
and <YourClass>.h
.
Start by opening the header file (the one ending in .h
). If the files aren’t already open, you can find them and double-click on them in the “Solution Explorer” sidebar. They’ll be in the Source
folder.
If you’re new to C++, classes are declared and defined in seperate files. The header file is where you declare a function, (or class or whatever) which lets the compiler know that it exists, including it’s name, return type, and parameters. This definition is called a function signature. The automatically generated class doesn’t have any functions in it, so let’s add a declaration. Inside the class body (between {
and }
), add the following:
The UFUNCTION
bit isn’t part of a normal C++ function declaration, it’s a macro which Unreal Build Tool uses to do some stuff behind the scenes, like finding the function and creating a blueprint node for it. You can add parameters to this macro call to do all sorts of things, but for now we’re just using BlueprintCallable
, which causes the Unreal Build Tool to automatically create a Blueprint node for the function.
The actual signature proper is static float SquareNumber(const float In);
. The static
keyword means that the function isn’t attached to instances of a class, it just exists all by itself. The next part is the function’s return type, which is what output the function returns when it’s done. In this case, this function will outupt a float. SquareNumber
is the name of the function, and const float In
declares a single parameter to the function, In
, which is a const float
. const
is a C++ keyword that means you aren’t allowed to change the value. The Unreal Build Tool uses the const
keyword to identify function inputs, while non-const
parameters that are references are outputs. We’ll get to that in detail later. The semicolon at the end marks the end of the function declaration. Now, we have to actually define the function in the source file.
Open the source file for your new class (the one ending in .cpp
). It should be totally empty except for a comment and a single line: #include <YourClass>.h
. This is an include statement, which lets the compiler find declarations in other files. In this case, we only need… the header we just wrote, so we can define the function from it.
Below that, add the following:
This looks a lot like our declaration from before, with a few differences. First is the UTutorialBPLibrary::
part before the function name. You should replace this with the name of your class. What this means is that we’re specifically defining the function SquareNumber
inside of UTutorialBPLibrary
. The ::
and the name before it specify a namespace, which means exactly what it sounds like, a space for names. Your class has its own namespace, that way we could define a different function SquareNumber
in a different class without causing any conflicts between the two. Here, we just want to specify our class’s function.
The other difference is that instead of a semicolon at the end, there’s a pair of braces. That’s where the actual function code goes. The part where it does the things and stuff. In this very simple example, I’m just writing a function that takes a float as a parameter, squares it, and returns the squared number. We can write this in one line: return FMath::Pow(In, 2);
. Here we see the ::
symbol again, this time because we’re referencing a function in FMath
, which is Unreal’s math library. The function is Pow
, which rases the first parameter to the power of the second, in this case, In
squared. Starting a line with return
means that we’re done, and we’re returning what comes after it.
And that’s all the code we need to make a custom Blueprint node! The UFUNCTION
macro in the header (specifically the BlueprintCallable
part) will tell the Unreal Build Tool to generate all of the other necessary code for us. So, compile the project! You can do this in Visual Studio, by right-clicking on your project in the “Solution Explorer” sidebar and selecting “Build”, or by clicking the Compile button back in the main editor.
Once the compilation is complete, go back to the editor and open up a Blueprint (make a new one if this is a fresh project). Try searching for your new node by name. If it doesn’t show up, it’s probably because hot reload didn’t work, because hot reload never works. Close the editor and reopen it.
There’s our node! If you hook this up, it should square the number:
But this node doesn’t really need Exec pins, right? The square node that Unreal already has doesn’t use ‘em. Let’s go back to the header file and take a closer look at those UFUNCTION parameters. Time to add some more bells and whistles.
I’ve replaced BlueprintCallable
with BlueprintPure
. This turns it into a “pure” function, which for blueprints simply means it won’t have Exec pins. The compiler doesn’t enforce much, but generally pure functions aren’t supposed to modify anything; you should use non-pure functions with Exec pins for that, in order to force everything to execute in the right order. In this case we’re just taking one number and returning another, so it’s a good idea to make it pure.
Next, there’s the meta
parameter. For some reason, some of these parameters are in a “meta” category, and so they have to go in here. I don’t really understand why; presumably it’s some technical implementation detail. Anyway, DisplayName
lets you define a nice display name for your node, including any special characters that aren’t valid in C++ function names. Using CompactNodeTitle
will make a node compact, which means it doesn’t have a title bar or names listed for pins, it just has one simple label in the center. The value for the parameter is the label that will appear on the compact node. Lastly, Category
lets you put your node in its own category in the node selector dropdown. You can use the vertical bar character (|
) to make nested categories, e.g. Utilities|Math|Float
would appear under Utilities > Math > Float. There’s a whole bunch of parameters you can use, including adding extra keywords to search for a node. You can read about them all here. After recompiling, here’s what the node looks like:
It’s compact! It’s pure! It…. looks kinda bad. The issue here is that the parameter input box overlaps the node’s label. There’s not a lot you can do to change the layout of a compact node, so to make things look prettier, I’ll change the parameter to a reference, which removes the ability to enter a value right on the node. In fact, I guess now is a good time to properly go over the different kinds of function parameters.
Here’s an example of more complex function:
I’ve split the parameters into multiple lines to keep it from getting too long. There’s a lot of little differences here. First, the function’s return type is now void
, which means it returns nothing. Well, actually that’s not true. This C++ function doesn’t return anything, but the node will return something, in fact it’ll return two things. Functions in C++ can only return one value, but nodes can return several. Making a node with multiple outputs can be accomplished through reference parameters. OutputOne
and OutputTwo
are the two outputs for this node, and they’re designated as reference parameters by the &
after the type. Any parameters that are non-const references will become outputs for the node (along with the C++ return value), and everything else will default to inputs. You “return” these values from C++ by simply assigning to the reference parameter, e.g. OutputOne = 5
. OutputTwo
is an AActor
, which means it gets passed around by pointer (thus the *
). Despite a pointer already being a kind of reference, output parameters still need to be C++ references, resulting in the somewhat silly AActor*&
.
I’ve made InputOne
a reference here (float&
), which removes the input box like I was describing for the other node earlier. But, since it’s const
it will still be treated as an input. I had to put InputThree
at the end due to a limitation in C++. You can declare optional parameters in C++, providing a default value for them if they’re omitted, but they have to go at the end of the signature. The Unreal Build Tool will automatically set up default values on a node from any optional parameters like this.
Here’s what this node looks like:
There’s the node, with the pin directions figured out by the compiler: three inputs and two outputs. The node is also pure, because of the BlueprintPure
, so no Exec pins. You’ll notice that the string input has the default value I specified, and that the float input doesn’t have an input box, since it requires a reference.
By the way, here’s what the Square node from before looks like without the input box in the way:
To round things out, I’ll walk you through one complete practical example: a function that checks whether an array is empty.
This one has some specifiers you’ve seen before, BlueprintPure, DisplayName, CompactNodeTitle, Category, and that Keywords specifier I mentioned earlier. It also has a new one: ArrayParm
(No, that’s not a typo (well not my typo at least lmao). It’s spelled “parm” as in parmesan… for some reason). This specifies any parameters (separated by commas) that should be treated as wildcard array parameters. Meaning, we’ll be able to hook up any kind of array to this node. The return type is a boolean (true or false), and the one parameter is a const reference to a TArray
of UProperty*
s. Why UProperty? That’s the base class for all of the blueprint parameter types, and it’s necessary for the wildcard pin to work.
The function definition is very straightforward:
Array.Num()
returns the number of elements in the array, and if that’s zero then the array is empty. You can put it all on one line like that because the compiler is smart enough to figure out that Array.Num() == 0
should be interpreted as a boolean.
Here’s what the node looks like:
As you can see, the input is an array pin, and it’s grey meaning it’s a wildcard and we can plug any kind of array into it. Once you connect one, it colors itself to match:
As an important sidenote, you can’t actually do anything with the array contents this way. To do that, you’ll need to specify a regular type for the array, or make a custom “thunk” function in order to get the necessary addresses to work with wildcard parameters, which I’ll cover in the next guide.
Now that you know how to make custom nodes, you can harness the power of C++ any time you need high performance or complex algorithms, and you can do it from Blueprints. If you’re feeling adventurous, check out the list of UFUNCTION parameters and try some of them out. Most questions you’ll end up having can be be answered with some googling, but if you have any trouble with this tutorial, let me know! (on mastodon, twitter, or by email) I’d be happy to help.
The next guide in this series will be a good deal more advanced, covering the creation of custom “thunk” functions in order to get addresses off of the Blueprint VM stack, to properly work with wildcard parameters.
See you next time!
]]>Note
I wrote this post a while ago, on the previous version of my site. I've included it here cause I think it's Pretty Alright.
It’s time for another Unreal Engine tutorial! This time, we’ll be making a component to let the player interact with an actor, two different ways.
This approach does require a tiny bit of C++, so for this guide I’ll make a new first-person C++ project.
Then once you’ve got your project opened up, under C++ classes in the content browser, right-click and create a new class. Derive it from SceneComponent and call it “InteractableComponent”.
The default code generated for this class has several things we don’t need, so remove just about everything from the class until we’re left with this in the header:
And the source file should be completely empty except for the include:
Now, we’re gonna add something called a “delegate”. Add this line outside the class definition, between the includes and the class:
A “delegate” is basically a template for a function with a given signature. The qualifiers “dynamic” and “multicast” are a bit outside the scope of this tutorial, but you don’t need to understand them for this. If you want to read about them on your own, look at this article in the docs.
DECLARE_DYNAMIC_MULTICAST_DELEGATE_OneParam
is a macro that defines the kind of delegate we need. FInteractableDelegate
is the name of the new delegate, UObject*
is the type of the one parameter, which is named Context
.
UE4 has several macros for defining delegates that are named as follows: DECLARE_< DYNAMIC_ >< MULTICAST_ >DELEGATE< _signature >
. For example, DECLARE_DELEGATE_RetVal
declares a regular delegate with a return value, DECLARE_DYNAMIC_DELEGATE_TwoParams
declares a dynamic delegate that takes two parameters. In this case, the delegate we’re declaring a dynamic multicast delegate that takes one parameter and returns nothing, so we use DECLARE_DYNAMIC_MULTICAST_DELEGATE_OneParam
.
Now that we’ve declared the delegate type, we’ll add one of these delegates to our class. Add these lines under public:
This adds an instance of our new delegate type to the class, called OnInteract
. UPROPERTY(BlueprintAssignable)
is a macro that makes the property visible and usable in blueprints. On a delegate, this means it will show up as an event in blueprints.
We’ve got one more thing to add to the header: a function to trigger the interactable. Add these lines just below:
This declares a function in our component called Trigger
. We’ll call this later to actually do the interacting. It takes one parameter, an optional object to use however you like in the event handler. UFUNCTION(BlueprintCallable)
makes the function callable from blueprints.
Now, we have one last thing to do: defining the function. In the source file, define the function:
All this function does is call Broadcast
on the delegate, which calls any event handlers that are bound to the delegate. It passes the context object through as well.
That’s it for the coding, return to the editor and compile your code.
InteractableComponent.h
InteractableComponent.cpp
If you just copy and paste this code into your project, make sure you change TUTORIALPROJECT_API
to match your project’s name, otherwise the compiler will throw a fit.
At this point, you should have a new component in the content browser:
If the code compiled successfully but the component doesn’t show up, something went wrong with hot reload and you’ll need to restart the editor.
Let’s make a blueprint to demonstrate how this component works. Make a new actor blueprint.
Open up the blueprint. I’ll be making a door for this example, but you can make whatever you like.
Now that my Very Good door is made, it’s time to make it interactable. Our new component should now show up in the components menu. Add one to the blueprint.
The component is a scene component, which means it has a position, but the position of the component itself doesn’t matter. However, since it’s a scene component, we can add child components to it. Add a box collision component as a child of the interactable.
Now, the position of this collider does matter. It will define the area we can interact with. Position it so it encloses the part you want to be interactable.
Now, with the interactable component selected, you should see a list of events in the details panel.
That OnInteract
event is the delegate we set up in C++. Click the + button to bind a handler to this event. There should now be a new event node in the event graph.
As you can see, our optional context object is available here for us to use in the interact event handler. In a real game you’d open the door here, or do whatever you need to for interaction, but for now I’ll just put a placeholder there.
That’s it for the interactable actor. Compile, save, and return to the main editor window.
I’ve add an instance of the door blueprint to the level, but it can’t actually be interacted with yet. First, we’ll need to do some setup in the player blueprint. But before that, we need to add an input binding for interacting. Open the project settings (Edit > Project Settings) and choose the Input page from the list on the left. Add a new action binding called Interact and assign a key to it.
Now open the Content/FirstPersonCPP/Blueprints/FirstPersonCharacter
blueprint.
Since there isn’t any custom functionality in this blueprint by default, the editor considers it a “data-only blueprint” and shows a condensed view of the blueprint’s fields. Click the link at the top to open the full blueprint editor.
Add an InputAction Interact node. This will be called every time we press the interact button.
Get the location of the camera. This is where the trace will start.
Now we’ll need to add to this position to get the end point for the trace. For this example I’ll use 200 units. Make a vector with that many units in the X axis.
But, this is the X axis in world space! In order to get 200 units in front of the camera, we’ll need to rotate the vector by the camera’s rotation:
Then, add that rotated vector to the camera position to get the end point for the trace.
Now, add a LineTraceForObjects node. This is what will perform the actual trace.
If you don’t know what this means, basically it follows a line from the start point to the end point, and returns the first thing it hits (if anything). Connect the start and end point nodes to the points we calculated:
Now drag a line out from the Object Types
node and add a Make Array node. This dictates what kinds of objects the trace will test for. Add WorldStatic
and WorldDynamic
.
That’s all the inputs we need hooked up, now let’s look at the outputs of the node. First, we’ll check the Return Value
node to see if we actually hit something. If this is false, the trace never hit anything, so we can stop there.
I’ve also broken out the Hit
object. This struct will contain data about the hit, if there was one. We’re interested in the Hit Component
value, which contains the particular component the trace hit. In this case, we’re looking for the box component we added to the door earlier. Try to cast the component to a BoxComponent:
If it is a box collider, we’ll want to check to see if the parent is an InteractableComponent. Get the Attach Parent and try casting it. If it is an interactable, trigger it by calling the Trigger
function we created earlier.
Phew! Finally everything is set up. Here’s where you could add a context object if you want, but you don’t need to give it anything. At this point, you can compile and save the player blueprint.
Run the level, point to your interactable, and press E. It should call the event handler you set up earlier.
Using a component for interactions as opposed to a parent class or interface, has a couple advantages. Unlike an InteractableActor class that interactables must inherit, interactable components can be dropped in anywhere with no changes to class structure. An interface could be dropped into any class, but with an interface you can only have one interactable per blueprint. If you want two interactables, or two hitboxes for the same interactable, you’ll need to use two separate blueprints. And speaking of, specifying the bounds for interacting is a lot harder with an interface, compared to just adding colliders to the scene.
Currently, this setup only works with box colliders, but it’s quite easy to add support for all three types. For some reason, there doesn’t seem to be a parent class for collider components, so you’ll have to check all three separately, like so:
This mouse-over technique doesn’t work quite as well in third-person. So we’ll use the same component but set up the colliders and player controller differently.
Make a new project, still C++ but using the Third Person template this time.
Add the same InteractableComponent class we wrote earlier. This time, I’ll make a simple overlay to appear when you’re able to interact. This isn’t a UMG tutorial, so I’ll skip the details here.
Make the interactable blueprint again, but this time make the box collider cover cover a large area in front of the object instead.
Hook up the interact event again too:
Set up an input binding again:
Now, go open the player character blueprint (Content/ThirdPersonCPP/Blueprints/ThirdPersonCharacter
). Instead of doing a line trace, this time we’ll be checking if the player itself is overlapping the hitbox for an interactable. For this, we’ll use the Get Overlapping Components node, which does exactly what it says.
To keep things tidy, let’s put things in a function. In the “My Blueprint” panel, click the plus button to add a function, and call it GetClosestInteractable
.
Why do we call it “get closest”? Because it’s possible that we could be overlapping the hitboxes for multiple interactables. In that case, we’ll want the closest one to get priority.
Inside the function. create a For Each loop to iterate through the components we’re overlapping.
We’ll do the same casting routine from the first-person example, for each element in the array.
However, this time we need to keep track of which one is closest. For that we’ll use two variables: ClosestDist
for the closest distance found, and ClosestComp
for the closest component itself. We’ll make them local variables since we won’t use them outside of this function.
ClosestDist
should be a float, and ClosestComp
should be an InteractableComponent. Make sure to clear ClosestComp
at the beginning of the function, since it’s possible we won’t find any interactables, and we don’t want to use a leftover value if there’s nothing to loop through.
Back inside the loop, after we’ve determined that we are overlapping an interactable’s hitbox, find the distance to the interactable like so:
Now we’ll check to see if either A) we haven’t found any interactables yet, or B) if we have found any, if this one is closer. If either of those conditions are true, we’ll save this one as the closest, as well as keep track of its distance.
Now, back to the loop node, when the loop is finished, we’ll have this function return what it found (or nothing), as well as a bool for whether or not it found anything. Add a new return node, and with the node selected, add two outputs in the details panel.
Found
is a bool, which indicates whether or not we found any interactables. Interactable
is either a reference to that interactable, or nothing if we didn’t find one. Hook up the return node like this:
Back in the event graph, add an input event node for the interact button again, and have it call our new function.
Now we’ll simply check if any interactables were found, and trigger the closest one if so.
Now if you compile, save, and run the game, you should be able to interact with objects when you’re standing in their hitbox!
But it’s not a great gameplay experience to just press E everywhere and hope something’s interactable. Remember that overlay widget I made earlier? I’ll use that to indicate when it’s possible to interact with something.
Here I create the widget on BeginPlay
, add it to the viewport, store it in a variable, and then hide it. Then, each tick, I call the function we wrote earlier, and depending on whether or not we’re overlapping any interactables, I either show or hide the widget.
Voilà!
Phew! That’s easily the longest guide I’ve written to date. I hope it’s helpful! If you have any questions, feel free to contact me (hello@trashbyte.io, cybre.space, twitter) and I’ll try to answer them! And if you enjoyed this guide, please consider throwing a few dollars my way (Patreon, Ko-fi). Your support means a lot to me. ♥
See you next time!
]]>Note
I wrote this post a while ago, on the previous version of my site. I've included it here cause I think it's Pretty Alright.
This is a hands-on tutorial about making sounds with modular synths. It’ll be a combination of an explanatory article and a tutorial. I won’t be providing audio samples here, so you’ll need to follow along on your own. Come on, it’ll be fun!
“Modular synths” are synthesizers made from discrete “modules” – individual components that perform a very specific function. They can be mixed and matched and interconnected in any way to make all kinds of sounds. Here’s what one looks like IRL:
I’d love for everyone to have access to this sort of thing here in meatspace, but the setup above is easily thousands of dollars. Luckily, there’s a completely free (and open source) software alternative available! It’s called VCV Rack and we’ll be using it for the rest of this article. (And for anyone who wants to use this with a DAW, there is a bridge plugin that can send audio and CVs back and forth!)
Start by downloading VCV Rack. It’s available for Windows, Linux, and MacOS. Once you’ve got it installed, open it up. You’ll see a blank rack like this:
Right click in an empty spot on the rack to bring up the module selector. We’ll start with a basic setup to get some audio out of this thing. You can type in the selector to filter it. Find the “Audio” module and add it.
This module provides audio inputs and outputs, for getting audio in and out of VCV Rack. Click on the black text displays at the top to select your audio subsystem and output device. If you have an ASIO driver, or another low-latency audio driver, installed on your computer, good! Select that. If not, just go with whatever is standard for your OS. Windows has WASAPI and DirectSound, either will work. Then select your output device from the second menu.
Once you’ve got the right audio device selected, green lights should show up next to the input and output jacks, according to how many channels your audio device has. Most likely, it has one stereo input and one stereo output.
Now we need some audio to hook up to it. But first, we’re gonna add an amplifier, to (ironically) make the signal quieter. The audio source we’ll be using will be full-volume, and I don’t want to hurt anyone’s ears. Right-click and add the “VCA” module. (That stands for Voltage Controlled Amplifier).
This is actually a dual VCA module, since there’s two seperate VCA circuits in it, the top half and the bottom half.
The knob controls the volume, and the four jacks are as follows:
Wait, what’s a CV? That stands for Control Voltage, and it’s the term for a signal used to control a parameter in the synth. In this case, the EXP and LIN inputs take an input and use that signal to control the volume. The LIN input affects the output linearly, for a linear signal input, while the EXP input adjusts it exponentially. The cool thing about modular synths is that there’s no hard distiction between audio and CV signals. You could use an audio signal as a CV, or vice versa. In this case, we won’t be using either CV input just yet. Just crank the volume way down and move on to the next module, which is the “VCO-1”.
VCO stands for Voltage Controlled Oscillator. This is your run-of-the-mill basic sounds source. It outputs a signal that “oscillates”, or cycles, at a certain frequency. It outputs 4 waveforms: sine, triangle, sawtooth, and square. It outputs all four of them at the same time; you can use any or all of them. A graph of the signals would look like this:
Now, let’s hear what these waveforms sound like! Drag a cable from the SIN output to the IN input on the VCA. Connect the OUT jack from the VCA to both input jacks on the audio module (left and right). Here’s a tip: when connecting an output to multiple inputs, drag from the destination, to the source. If you try to connect from the output it’ll move the existing connection. You’ll see what I mean when you start fiddling with the cables. Here’s what our setup looks like:
If you turn up the knob on the VCA, you should hear a constant tone from your speakers. If you don’t hear anything, try changing the output devices until you hear something.
Now try listening to the other waveforms! Connect the TRI, SAW, and SQR outputs to the VCA and listen to each of them. NOTE: the sine output is the softest sounding by far. Each other signal will be harsher, so adjust your VCA accordingly.
Let’s add a scope module so we can visualize these waveforms. Add the “Scope” module, and hook the output of the VCA to the “X IN” input.
As you change the waveform or volume, you’ll see the signal on the scope change. Try turning that big FREQ knob on the VCO. Predictably, it changes the frequency (pitch) of the oscillator. You can see the waveform on the scope looks slightly different from the chart I showed above. That’s because the oscillator is in “analog mode”. That means it’s emulating the kinds of distortion that usually occur in analog oscillator circuits. If you want the pure digiatal, “perfect” waveforms, click on the ANLG/DIGI switch on the upper left of the VCO. Some people (myself included) think the analog versions sound a little nicer. You can decide for yourself once you’ve done some more fiddling.
Now, obviously, turning that knob to select the frequency isn’t an ideal solution. CVs to the rescue! Right-click and add the “MIDI-1” module.
This module creates CV signals based on MIDI input. MIDI stands for Musical Instrument Digital Interface, and it’s a protocol for controlling electronic instruments. If you’re already the music-savvy type, you might have a MIDI keyboard already. In that case, hook it up to your computer and select the device on the module. If not, don’t worry, we can still make some MIDI signals in software.
At the time I’m writing this, VCV Rack doesn’t have any way to create MIDI signals using your computer keyboard. So, we’ll use a different solution. There’s an easy way and a hard way.
The simple way is to install the ImpromptuModular plugin. Jump to the end of this article and follow the instructions for installing the plugin, then add the “Twelve-Key” module. From here on out, ditch the MIDI module and use the CV and GATE jacks on that module instead. You can click on the keys to generate signals, and use the + / - buttons to change the octave.
Alternatively, for a more robust solution, download VMPK. It’s a freeware program for creating MIDI signals. It too is available on Windows, Linux, and MacOS. Now, we also need a way to get those signals into VCV Rack. On Windows, download and install LoopBe1. It’s a virtual MIDI driver that will allow us to get the MIDI signals from VMPK to VCV Rack. On Linux, try using JACK. On MacOS, uh, you’re on your own, sorry. I’m not really familiar and I don’t have a Mac handy.
Using VMPK and LoopBe1, open VMPK and close and reopen VCV Rack. (VCV Rack should save your current rack setup and reload it when you open it next). In the MIDI module, select “Windows MIDI” and “LoopBe Internal MIDI 0”. You should now be able to click on keys in VMPK or type on your keyboard with VMPK selected to send MIDI signals to VCV rack. Now let’s get back to actually hooking those signals up.
Okay, back to business! There’s a lot of signals on the MIDI module, but we’re mainly interested in CV and GATE. CV is a signal representing the pitch of the current note played, and GATE is signal representing whether a key is pressed or not. Hook up the CV signal to the V/OCT input on the VCO. V/OCT refers to a common standard in hardware modular synths, where 1 volt equals one octave. So a +2 volt signal would correspond to an adjustment of +2 octaves. Now, hitting keys should change the pitch of the oscillator. We’re getting closer!
Now, we need a way to actually turn the oscillator off when we aren’t hitting a key. We’ve already got a GATE signal, and a VCA which can make things quieter, but we’re going to use a special module to give us more control over how the sound changes when we press a key. Add an “ADSR” module.
In case you haven’t noticed yet, modular peeps love their acronyms. ADSR stands for Attack Sustain Decay Release, which are the four parameters of an “envelope”, which is a curve dictating how a sound changes when you press a key. Here’s a graph of the curve:
“Attack” is the fade-in time when you press a key. The longer the attack, the slower the note fades in. At zero attack, the note comes on instantly.
“Sustain” is the one parameter here that isn’t a duration – it’s the volume that the note holds at. “Decay” is the tine it takes to fade to that amount. With sustain and decay at max, the note just holds at full volume. With sustain at zero, the note fades out according to the decay time.
“Release” is the time the note takes to fade after you release the key.
Together, these parameters define an “envelope”. This signal can be used to control any parameter. It also can be triggered by things other than a key press, but that’s what we’re using it for here. We’ll use it to control the VCA. Connect the OUT jack on the ADSR to the EXP input on the VCA. Then, connect the GATE output from the MIDI module to the GATE input on the ADSR. That way, the GATE signal will “trigger” the ADSR – causing it to generate an envelope controlling the VCA.
Now, when you press a key, you should hear a note fade in and out, according to the parameters on the ADSR. Fiddle with them until you’re comfortable with how they work.
Now that we’ve got a basic note set up, lets make it a bit more interesting. Add a “VCF” module.
VCF stands for “Voltage-Controlled Filter”. This is used to “filter” a sound, which means to remove certain frequencies. This filter is a combination low-pass and high-pass filter, outputing both at once. A high-pass filter is one that allows high frequencies through, but filters out lower frequencies. A low-pass filter is the opposite – it filters high frequencies, letting the lows pass. The point where the filter starts to filter these frequencies, as well as the “sharpness” of the filter, are adjustable.
Connect the output of the VCA to the IN jack on the VCF, and the VCF’s LPF jack to the output & scope. This will make the signal pass through the (low-pass) filter before going to the output.
Choose a harmonically-rich waveform like the SAW or SQR from the VCO, that way you can actually hear what the filter is doing. Turn the FREQ knob all the way down. Since we’re using the low-pass output, this will filter out basically everything. As you turn the knob back up, it will let more and more frequencies through, from low to high. Listen carefully to how this sounds. If you want, set the parameters on the ADSR to these:
This will give you a nice constant tone to mess with the filter. You can also change the RES knob on the filter, which control’s the filter’s “resonance”, which basically means how “sharp” the filter is. Turn it up and listen to how the filter sounds, especially as you change the FREQ.
If you want, go ahead and try the HPF jack on the filter. This is the high-pass output, and is basically the opposite of the low-pass. It should be pretty obvious when you hear it.
Now, let’s use another ADSR for the filter. Right-click on the ADSR and choose “Duplicate”. Connect the MIDI GATE to the new ADSR’s GATE, and the ADSR’s OUT to the VCF’s FREQ jack. Unlike the VCA, the filter has a knob to control the amount the CV affects the filter frequency. If we turn the knob up, the envelope increases the filter frequency, if we turn it down, it does the opposite. Turn the FREQ CV knob all the way up, and the FREQ knob all the way down. This way, the filter will start “closed”, and the ADSR will “open” it up. Now fiddle with with ADSR and VCF settings and see how they affect the sound!
At this point, I’m sure you have some ideas about things to explore. Go for it! With all the different things to connect, there’s an infinite amount of sounds to explore. If you don’t know what any of the knobs or jacks do, just google the name of the module and the name of the control. Unfortunately the official documentation is very lacking (most modules are completely missing), but googling “VCO SYNC jack” will probably bring up what you need.
Oh. And here’s where it gets really fun. VCV Rack is open source, and lots of users have created their own modules for it! You can download them here. Make an account, and log in on the website and in VCV rack. Click the “+ Free” button on any free plugins that catch your fancy, then click “update plugins” at the top of the VCV Rack window. The plugins will sync automatically.
Here’s a few of my suggestions:
Haha, that’s right! I’m almost certainly going to be writing my own modules for this, so keep an eye out!
Good luck, and have fun making music!
i did make a few modules, but they’re not currently available online anywhere. feel free to send me a message at hello@trashbyte.io or cybre.space if you’re interested and i’ll release em
Note
I wrote this post a while ago, on the previous version of my site. I've included it here cause I think it's Pretty Alright.
In my last post on materials, I explained what a material is, and the basic paraemeters that describe one. In this post, I’ll be demonstrating how to make your own materials in Unreal Engine!
Here’s what we’ll be making:
Start by downloading and installing Unreal Engine. Create a new blank project.
We don’t need the starter content, so don’t include it. Or do. I’m not your mom. But all of the content in Unreal Engine is already accessible from any project, so we won’t need to include the starter content here.
Once you’ve opened up your new project, right click inside the content browser panel at the bottom of the screen and select Create Basic Asset > Material.
Double-click on the new material to open the material editor.
On the right side is a preview of the material applied to a sphere. Below it is the inspector, which shows the details of whatever node we have selected, or the options for the whole material if nothing is selected. In the center is the node editor, where we’ll be building the material. Finally, on the right is a searchable list of all available material nodes.
Right now, there’s only one node: the output node.
Each of these points is one property for the material. Some of these are greyed out, like opacity. That’s because they aren’t applicable to the current material. Remember in the last post, when I explained that materials are opaque by default? Since this is an opaque material, we don’t use the opacity property. You can still hook stuff up to it, it just won’t do anything. There are also a few properties I didn’t cover in the last post, but we won’t be using any of them in this tutorial. I’ll explain them in another article later.
In the bottom-right corner of the preview pane, there are a few icons. These buttons change the preview model used to display the material. You can even supply your own model using the far-right button. Since we’re making a material for flat tiles, select the cube:
You can click and drag in the preview window to rotate the view, and use the scroll wheel to zoom.
Right-click anywhere inside the node editor to bring up this dropdown list:
This is a list of available nodes, the same list that’s visible on the right side. You can just start typing to filter the list down, and hit enter or click on an item to add it. Alternatively, you can drag an item out of the list on the right side panel to add a node. From this point on in the tutorial, I’ll just say to “add X node”, and you can use either the right-click menu, or the panel on the right, whichever you prefer.
Start by adding a “Constant” node:
This is probably the simplest node. It reperesents a single value, which never changes (a “constant” value).
With the constant node selected, in the inspector on the right, we can see the properties for the node.
As expected, there’s just a single value for the node. There’s also a space where you can write a comment that will show up above the node:
Leave the value at 0 for now, and click and drag the output of the constant node, over to the input for “Roughness”, and release.
With these points hooked up, the value for Roughness is now set to the constant value 0 – it’s zero across the whole material. In the preview pane, the material will automatically update and become glossier, since we reduced the roughness to 0. Try changing the constant to 1 and see what happens!
Add another constant for Metallic. There’s a shortcut for adding constants: hold the 1 key and click to add a constant node. Hook that constant up to the Metallic input. Your nodes should look like this now:
Wait, why was the shortcut for that to hold 1? Well, try holding 3 and clicking.
The answer is that the constant node is a single value, a one-dimensional vector, if you will. This new Constant3Vector node is a 3D vector – a fancy way of saying a group of 3 values. Since we’re making materials here, Unreal treats this Vector3 as a color – one value for red, one for green, and one for blue. (Computer screems make colors by combining red, green, and blue light in various amounts.)
Hook this node up to the Base Color node and play with the values in the inspector on the left. You can click on the colored bar to open a color picker. The material will change color to match, and the box on the node itself will show the color too. If desired, you can click the arrow in the upper right of the node to collapse it and hide the preview. You can also expand the single constant nodes, although it’s not a very useful preview since it’s just a single value. Most nodes have an optional preview like this.
If you set any of these values to outside of the 0..1 range, they’ll be clamped inside that range. (Well, most of them will. Emmisive, for example, is unclamped, and you can even allow special behavior for negative values.) This is true for the final output nodes, but not the intermediary nodes. The ability to have values outside of 0..1 for calculations inside the material is very important.
Let’s add a texture map. Here’s a tile mask and normal map you can use:
Just right-click and Save As. Drag these images into the content browser in the main Unreal window to add them as assets. Then drag those assets onto the material node editor to add them as TextureSample nodes. Connect them to the Base Color and Normal inputs like so:
The TextureSample node has a whopping 5 color-coded outputs. The top one is a Vector3 containing the color of the texture. This is the one you’ll use most often. The red, green, and blue outputs below are the respective single color components of the image. The last, grey output is the “alpha channel” for the texture, if there is one. This is usually used for transparency in normal images, but for use in a game engine, this could be used for anything. Sometimes albedo maps will use the alpha channel for roughness information.
After hooking up this node, you’ll notice the material now has a base color pattern matching the mask texture, and the normal map added a subtle bit of depth to the cracks.
On the left side of the TextureSample nodes are two inputs: UVs and Tex. Tex is an input for a texture object – basically just another way to specify what texture to use. The UVs input is where you can specify the UV coordinates to sample the image at.
The way UV coordinates works is, the 3D model has coordinates for each vertex of each face. These coordinates map to a location on the texture so it can be displayed properly on the model. For example, this cube’s top face has four vertices – each corner. In this case, the UV coordinates for these vertices just map to the four corners of the texture, so the texture is streteched perfectly across the surface.
Now, we can fiddle with these coordinates to achieve certain results. By default, when the UVs input is unconnected, it’s hooked up to the model’s first UV channel automatically. (Models can have multiple sets of UVs for more complex materials.) Let’s manually add a TextureCoordinate node. The shortcut for this is to hold U and click. Hook it up to both UVs inputs.
This setup is the same as not hooking up anything – the textures are using the model’s first set of UV coordinates. Now, let’s do some math! Add a Multiply node.
This node does exactly what you think. It has two inputs and one output – the output is the two inputs multiplied together. Add a constant node, and hook the TexCoord node and the constant to the inputs of the multiply node, and the output to the UVs of the textures.
Now the model’s UV coordinates will be multiplied by the value of the constant. If you set it to 2, now the texture shrinks on the model, and is displayed tiled, 2x2. The reason for this is that the original upper-left corner of the model had a UV coordinate of (1,1), meaning the upper-left corner of the texture. We’ve multiplied that value to (2,2), which is outside the bounds of the texture, but it just repeats forever, so we just get more texture. Here’s a visualization:
If you don’t totally understand what’s going on here, just think of multiplying the UVs as scaling the texture on the model.
Now, you can do math with any of these nodes, not just the UVs. Let’s add some color. Make another Constant3Vector node and multiply the output of the mask texture with it, routing the output of the multiply to Base Color.
Since all these colors are just values between 0 and 1, we can do math with them quite easily! In this case, the mask texture is just white (1) in most places, with a black (0) grid. Multiply that by the provided color and, 1 × color = color
and 0 × color = 0
. Math!
Let’s add some noise. Add another TextureSample node (hold T and click). Select the node, and in the inspector, find the “Texture” value. Click on the dropdown to open a list of every texture available in this project.
Unreal includes a few noise textures by default. You could select them from this list to use one. However, I don’t quite like how they look for this case, so I made my own noise texture:
Again, right-click and Save As and add it to the project. This time, select the new TextureSample node, find the “Texture” parameter again, and find the new noise texture in the dropdown menu. Remember that you can type to filter the list.
Hook up our scaled UVs to this texture, and connect the output to the Roughness input, replacing the constant node we were using before.
This adds an interesting, dirty, wet look to the tiles.
Now, I want to be able to tweak the intensity of this effect, but making changes in the material editor causes the material to recompile every time, which takes some time to update. Now is a good time to cover a powerful feature of Unreal Engine: Material Instances.
This is not only a standalone material, it can also be used as a template for multiple different materials, with tweakable parameters! Let’s add a parameter. Right-click on the constant attached to the UV multiplier and choose Convert to Parameter.
This makes the constant into a parameter for material instances. It’s still constant for the whole material, but we can set it for each instance. Here’s the inspector for the node:
Name the parameter “UVScale”. The exact name isn’t important, it’s just the name we’ll see in the material instance editor. Group and Sort Priority can be used to group the parameters and control their order. Below that are the default, minimum, and maximum values. Set the minumum to 0.1 and the maximum to 10. We’ll see what this means in a moment.
Save the material and return to the main Unreal window. Right-click on the material asset and choose “Create Material Instance”.
This creates an instance of this material. It’s basically the same underlying material, but we can alter any parameters we’ve made. Double-click on the material instance to open it.
The material instance editor is quite simple. It’s just a list of parameters on the left, and a huge preview window to the right. You can also select the preview model in the bottom-right corner here as well.
As you can see, our parameter UVScale is here! It’s disabled by default, so it uses the default value from the base material. Click the checkbox to override the default value, and drag the slider to change the UV scale. As you can see, the change is instant now, instead of having to wait for the material to recompile! Also you can see what “Slider Min” and “Slider Max” from before meant. Single constant parameters have a slider that you can use to quickly change the value. Note that the minimum and maximum values aren’t enforced, that’s just the range of the slider. You can click on the slider and type in any value you want.
Let’s go back to our base material and add some more parameters. In the inspector for the material instance editor, you can see the “parent” material. Just double-click on the thumbnail to open the parent material again.
Right-click on the Vector3 parameter for the Base Color, and convert it to a parameter too. Call it “TileColor”. Note that Vector3 parameters don’t have options for a slider, since they won’t have a single slider in the material instance editor.
Now, copy and paste that parameter node, and name the copy “GroutColor”. Now, make a new node, either by right-clicking or using the side menu. The node is called “LinearInterpolate”.
Linear interpolation is just a way of blending between two values. You provide two inputs, A and B, and the Alpha value controls where between the two you want the output to be. “Interpolate” means to blend between two values according to some function, and “Linear” just means the Alpha maps linearly to the blending, as opposed to exponentially or by some other function. So if Alpha is 0, the output is completely the A, if it’s 1 the output is completely B, and if it’s 0.5, it’s an even mix of the two. Any other value will be somewhere between the two.
In this case, we’ll be using the mask texture as our Alpha, to interpolate between our tile and grout colors. Hook up the GroutColor node to A, the TileColor to B, and the mask texture to Alpha. Hook the output up to Base Color. Delete the old multiply node we were using there.
Now, where the mask texture is 1 (white), the base color will be B, which is TileColor, where the mask is 0 (black), it’ll be A, our GroutColor. And it will fade smoothly in between according to the mask.
Save the material and return to the instance we made before. Our two new parameters show up now, and we can change the tile and grout color in here with immediate feedback.
Oops, I’m just now remembering that the reason I explained parameters and instances was to tweak the Roughness. Let’s do that now. Add these nodes and hook them up as follows:
(That’s two constant parameters hooked up to a lerp, with the noise texture as the Alpha). Note that I’m using the red channel of the noise texture. The Alpha input to the lerp node must be a “scalar value”, which is another way of saying a single numeric value. Colors are Vector3s, which are 3 numbers in one, so we can’t route that into Alpha. Instead, we just grab one of the channels – since the texture is black and white, all the channels are the same, so I just picked red.
Now, the noise texture will blend the roughness between the values supplied by the parameters. If we set them to 0.5 and 0.75, then where the texture is black would be roughness 0.5, white would be 0.75, and the greys would be values in between. Functionally, we’re remapping the noise texture from (0,1) to (min,max). Now we can tweak the effect more carefully. I’ll also add a parameter to exponentiate the mask value:
Since colors are just numbers, we can do any kind of math with them, including exponentiation (powers/roots)! Here’s what the noise texture looks like when squared and square-rooted:
Head back to the material instance and tweak the exponent. Hopefully it should be easier to understand when you see it demonstrated.
We’re almost done with this one! If you look closely, you can see that the noise texture is affecting the roughness inside the grout, whereas I want the grout to always be rough. The fix is fairly easy, just multiply the roughness by the mask texture, right? Well, the mask texture is actually backwards from what we want. This would make the grout roughness 0, when we want roughness 1. But we can’t just invert it and multiply, because that would make everything else zero.
What we’ll do is invert the texture so the grout lines are white, then add it to the roughness. This will result in some roughness values greater than 1, which shouldn’t be a problem since Unreal clamps roughness automatically, but we’ll add a clamp node just to be safe, which clips the values to keep them inside a specified range.
The invert node is called “OneMinus”, since it just performs 1-x
on the input. This is different from multiplying by -1, but for values between 0 and 1, the result is the same. Notice that I expanded the OneMinus and Add nodes by clicking on the triangle. This is one instance where the visualizations are helpful. The clamp node is just called “Clamp”, and the default values for the extents are 0 and 1, which is exactly what we want.
Whew! That’s pretty complicated. When the node graphs start getting big, you can use reroute nodes to keep things tidy. These are just points you can use to control where the connecting lines go. To add a reroute node, double click an existing line, or right click and select Add Reroute Node (type to filter). You can also drag a line out from a node and release to open the menu, then start typing to add any node, including reroute nodes.
Here’s our final graph:
Try making several instances and tweaking the values to make different tiles!
]]>
Note
I wrote this post a while ago, on the previous version of my site. I've included it here cause I think it's Pretty Alright.
This is the first in a series of posts explaining how 3D rendering works. These posts are designed to help new game developers understand the terminology, as well as the underlying concepts, as quickly as possible. They are designed for readers that plan on using a game engine (like Unity or Unreal Engine) that deal with the nitty-gritty stuff for them. You won’t find complex matrix transformations or raytracer implementations here. You’ll have plenty of time to read about all that on your own if you’re interested. For now, we’ll get started with the basics.
If you’ve dabbled in game dev at all, you’ve probably heard of something called a “shader”. This is a piece of code that was written to run on your video card, in order to draw graphics on a screen. A simple shader might look like this:
#version 410
layout (std140) uniform Matrices {
mat4 projModelViewMatrix;
mat3 normalMatrix;
};
in vec3 position;
in vec3 normal;
in vec2 texCoord;
out VertexData {
vec2 texCoord;
vec3 normal;
} VertexOut;
void main() {
VertexOut.texCoord = texCoord;
VertexOut.normal = normalize(normalMatrix * normal);
gl_Position = projModelViewMatrix * vec4(position, 1.0);
}
If you don’t understand any of that, don’t worry! Enough tools exist that you probably won’t ever need to write shader code like this, at least not until you already know what you’re doing.
Now, I’m going to stop talking about “shaders”, because most modern tools abstract these shaders away into “materials”. Instead of reasonaing about rendering in terms of programs, like a computer might, a material describes the physical properties of a thing you’re rendering. What color is it, how does it reflect light, how opaque is it, etc. A single material may be made up of many shaders under the hood, but we’ll let the tooling deal with that.
Here we’ll define the various physical properties that make up a material definition.
This one is pretty straightforward, it’s the basic surface color of the material.
A value between 0 and 1 that describes how “shiny” a material is. At high values, the material is “rough” and so most light is absorbed or scattered and little is reflected. At low values, the material is “smooth” and reflects most light.
This one is tricky. The reflected color of a metal is defined by the angle between the viewer and the light source. Unlike “specular highlights” that show up as a bright spot like on the low-roughness example above, the brightness of metallic sheen is based on the viewing angle.
Basically, metals reflect light differently from non-metals. The “metallic” value controls how “metal-like” a material is, where 0 is regular, non-metal reflections, and 1 is fully metallic reflective behavior.
Here’s a comparison of two white materials, with roughness 0, one with metallic 0 and one with metallic 1.
Here’s a comparison with a red albedo value instead.
Opacity describes how transparent a material is. At 0, the material is fully transparent, and therefore invisible. At 0.5, it’s half-transparent and see-through. At 1 it is fully opaque and cannot be seen through at all.
The emissive value controls how much light a material emits. By default, this is zero, since most materials don’t glow. However, you can set this value to greater than zero to make a material emit light. (Note that this light often won’t be cast onto other objects in a game engine, as calculating bounce light from emmisive materials is very computationally expensive.)
Here are a few black materials with various emissive values. On the left is no emissive value, on the middle is fully red, and on the right is a value of red greater than 1. This may do nothing depending on the engine you’re using, but many support emissive values greater than 1, which just means they emit more light.
Notice how the middle one is bright, but not really brighter than if it were just red. Keep in mind its albedo color is black. The one on the right, by contrast, is glowing brightly, with light bleeding around the edges due to bloom. Notice however, that the light isn’t being cast onto the floor as if it were a point light.
This parameter controls the colors scattered by a phenonenon known as “subsurface scattering”. Some materials, like human skin, scatter light inside the substance. This can be seen by shining a light through your hand. The light will spread out through the skin under the surface. You’ll notice that even if you shine a white light, the light scattered is always red. This is because skin absorbs most frequencies of light, but scatters red light beneath the surface.
The “subsurface color” parameter affects the color of light that is scattered beneath the surface of the material. The example on the left below has no subsurface color, meaning no light is scattered beneath the surface. The material on the right has a red subsurface color, so red light is scattered.
In a Metallic/Roughness PBR workflow, the specular value is not used for metals. In non-metals, it can be used to tweak the reflectivity of a material. In Unreal Engine, it is 0.5 by default. which works for most solid materials. If the specular value is ever used, it’s usually set by a texture map, which I’ll explain below.
These don’t make much sense as a single value, so I’ll explain it below, once we cover texture maps.
Here’s where things get interesting! Instead of using a single value to describe an entire material, we can use special “textures”, which are images describing the various values for a certain property. “Texture” is just the name for an image when it’s used in rendering software. When referring to textures that control parameters in a shader or material, they are usually called “maps”, e.g. “albedo map”.
Here’s an example of an material with an “albedo map”. On the left is the texture used, and the resulting material is on the right.
As you can see, the object has different albedo values across its surface, as defined by the albedo map.
Roughness maps are used to describe how an object’s roughness changes across its surface. Think “shiny in some places and not in others”.
Rendering transparency is gernerally pretty computationally expensive (since you have to render both the object itself and whatever is behind it), so most rendering engines don’t by default; they assume most objects are opaque. If you want to render an object to be see-through, there are two different ways to do it: translucency and opacity masks.
Translucency is the “normal” way to render transparency:
It’s best to avoid this kind of transparency unless it’s necessary for a certain effect, as it is expensive to render.
The other kind of “transparency” is with opacity masking, or “cutout transparency”. In this mode, you supply a “mask” texture that defines what parts of the material to draw:
Note that in this mode the material can only be fully opaque or completely invisible, nothing in between. This is so that the renderer never has to draw both the foreground and the background on top of each other, like with partial transparency. It’s useful for things like leaves on a tree, where you can supply the shape of the leaf as an opacity mask and just draw a single rectangle with the shape of the leaf masked out.
Metallicity and subsurface color are usually constant for a certain material. One exception might be if you used one material to render an object with multiple different surface types, but doing so sort of goes against the whole concept of materials. Occasionally using a metallic map can make objects look more realistic, especially things like rocks that actually have varied metallicity in real life.
Okay, this one is really cool. Normal mapping is a technique used to “cheat” extra surface detail into an object, without increasing the number of polygons in a model. Basically, you create a “normal map” that describes the slopes along a surface, and the renderer adds some shadows in certain places to fake depth. The map describes what’s known as a “normal”, which is a vector describing the direction of a face. In simple terms, the red channel describes different surface angles along one axis, and the green channel along the other axis. The blue channel is usually ignored, since we’re only describing a surface in two dimensions.
Here’s an example from wikipedia (courtesy of Julian Herzog):
On the left is a 3D model with actual shapes in the model itself. In the middle is a normal map representing the surface angles, or normals, of that model. On the right is a flat plane, rendered using that normal map. As you can see, even though it’s just a single flat plane, the shading creates the illusion of extra depth. For shallow details, this works just fine and can help recreate fine details while still maintaining a low polygon count on the actual model.
Ambient occlusion is a rendering technique that mimics the way light gets trapped inside narrow spaces. Here’s a real-life example:
In the tight space between the egg and the table, light gets trapped, so there’s a darker shadow there. General ambient occlusion is handled automatically be the lighting engine, but you can provide an AO map that describes places on the material that you want to manually add extra shading. This is similar to a normal map except it’s one-dimensional – it just describes the shading.
Here’s an example of an AO map used to add shading detail to a face:
A specular map is sometimes used with non-metals to change the specific reflectivity of a material. I won’t cover specular maps right now because it’s a more advanced topic, and generally unnecessary in a Metallic/Roughness PBR workflow. I may cover later it in an advanced rendering article.
Now I’ll cover a few example materials, so you can see how all the pieces fit together.
Here’s an example of a wood floor material, adding one map at a time to show how each one affects the surface.
The albedo map adds the color of the wood, but it still looks unnatural without detail. The roughness map adds areas that are smooth or rough, which immedately improves the look. Finally, the normal map adds the cracks between boards and small surface details.
This one includes a metallic map, which is fairly rare. In this case it’s because the rocks are partially metallic, so varying the metallic value makes them look more realistic.
The details in this one are pretty subtle, but pay attention to how the metallic map makes the rocks look a bit more metallic and shiny, while keeping the dirt between them dull.
Here’s an example of a leaf material that uses opacity masking.
On the left side are the albedo map (top) and the opacity mask (bottom). On the right is the material rendered without the mask (top) and with the mask (bottom). Using masked opacity as opposed to full translucency is especially important for leaves since you usually render hundreds of them at once. The added cost of full translucency would add up quickly.
Whew! Hopefully that helps explain how material-based rendering works. I’ll probably be writing more posts soon, including a tutorial for Unreal Engine’s material editor, so you can make your own materials. If there’s something here you don’t understand, please email me at hello@trashbyte.io
or message me on cybre.space (@trashbyte)! I wanna make sure these guides are easy to understand, and I’m happy to make edits to make them clearer.