URL Encoding Explained: Why and How Special Characters Are Escaped

What Is URL Encoding?

URL encoding, formally known as percent-encoding, is a mechanism for encoding characters that have special meaning in a URL or that are not allowed in a URL. Each encoded character is represented as a percent sign (%) followed by two hexadecimal digits representing the character's byte value in ASCII or UTF-8.

For example, the space character (ASCII value 32, hex 20) is encoded as %20. The ampersand character (ASCII value 38, hex 26) is encoded as %26. The at sign (ASCII value 64, hex 40) is encoded as %40.

URL encoding exists because URLs can only contain a limited set of characters from the US-ASCII character set. Characters outside this set — including Unicode characters, whitespace, and many punctuation marks — must be encoded before they can be safely included in a URL.

Why URL Encoding Is Necessary

URLs have a defined structure specified by RFC 3986. Different parts of a URL (scheme, authority, path, query, fragment) use certain characters as delimiters:

https://example.com:8080/path/to/page?key=value&other=data#section

• :// separates the scheme from the authority
• : separates the host from the port
• / separates path segments
• ? marks the beginning of the query string
• & separates query parameters
• = separates parameter names from values
• # marks the beginning of the fragment

If your data contains any of these characters, the URL parser would misinterpret the URL structure. For example, if a search query contains an ampersand — like searching for "Tom & Jerry" — the unencoded URL ?q=Tom & Jerry would be parsed as two separate parameters: q=Tom and Jerry= (with an empty value). The correct encoding is ?q=Tom%20%26%20Jerry, which preserves the intended meaning.

Beyond structural ambiguity, URL encoding also ensures that URLs remain safe for transmission across different systems, protocols, and networks that may not support the full range of characters.

RFC 3986: The Rules of URL Encoding

RFC 3986 ("Uniform Resource Identifier: Generic Syntax"), published in 2005, is the definitive specification for URI syntax and encoding. It defines which characters can appear in a URL without encoding and which must be percent-encoded.

Unreserved Characters (Safe — Never Need Encoding)

These characters can appear in any part of a URL without encoding:

• Letters: A-Z, a-z (52 characters)
• Digits: 0-9 (10 characters)
• Special: - (hyphen), . (period), _ (underscore), ~ (tilde)

In total, there are 66 unreserved characters. These characters are always safe and should never be percent-encoded. Encoding them is technically valid but unnecessary and makes URLs harder to read.

Reserved Characters (Delimiters — Context-Dependent)

These characters have special meaning in URLs and serve as delimiters:

• General delimiters: : / ? # [ ] @
• Sub-delimiters: ! $ & ' ( ) * + , ; =

Reserved characters must be percent-encoded when they appear in a context where they are not serving their reserved purpose. For example, / does not need encoding when separating path segments, but it must be encoded as %2F if it appears within a single path segment or as part of a query parameter value.

All Other Characters (Must Always Be Encoded)

Any character not in the unreserved or reserved sets must be percent-encoded. This includes spaces, control characters, non-ASCII characters (Unicode), and characters like <, >, {}, |, \, ^, and `.

Common Percent-Encoded Characters

Here is a reference table of the most frequently encoded characters:

• Space — %20 (or + in form data)
• ! — %21
• # — %23
• $ — %24
• & — %26
• ' — %27
• ( — %28
• ) — %29
• * — %2A
• + — %2B
• , — %2C
• / — %2F
• : — %3A
• ; — %3B
• = — %3D
• ? — %3F
• @ — %40
• [ — %5B
• ] — %5D

You can use our URL Encoder/Decoder tool to quickly encode or decode any string.

UTF-8 and Multi-Byte Encoding

Modern URLs need to support characters from every language in the world — Chinese, Arabic, Cyrillic, emoji, and everything in between. URL encoding handles this through UTF-8 multi-byte encoding.

The process works in two steps:

1. Encode the character as UTF-8 bytes. UTF-8 represents characters using one to four bytes. ASCII characters use one byte, most European and Middle Eastern characters use two bytes, most Asian characters use three bytes, and emoji and rare characters use four bytes.
2. Percent-encode each byte. Each byte of the UTF-8 sequence is independently percent-encoded.

For example, the German letter "u with umlaut" (the character at Unicode code point U+00FC) is encoded in UTF-8 as two bytes: 0xC3 0xBC. In a URL, it becomes %C3%BC.

A more complex example: the Chinese character for "dragon" (Unicode code point U+9F99) is encoded in UTF-8 as three bytes: 0xE9 0xBE 0x99. In a URL, it becomes %E9%BE%99.

Emoji can produce even longer sequences. The smiling face emoji (U+1F604) is encoded as four UTF-8 bytes: 0xF0 0x9F 0x98 0x84, resulting in %F0%9F%98%84 in a URL.

This is why it is critical that both the encoder and decoder agree on using UTF-8. Older systems that used Latin-1 or other single-byte encodings would produce different byte sequences, leading to garbled text. RFC 3986 and modern web standards mandate UTF-8 for percent-encoding.

encodeURI vs. encodeURIComponent in JavaScript

JavaScript provides two built-in functions for URL encoding, and choosing the wrong one is a common source of bugs. Understanding the difference is critical.

encodeURI()

encodeURI() is designed to encode a complete URI. It encodes all characters that are not valid in a URI except for the reserved characters that serve a structural purpose: ; , / ? : @ & = + $ #

Example:

encodeURI("https://example.com/path/to page?q=hello world")

Result: https://example.com/path/to%20page?q=hello%20world

Notice that the ://, /, ?, and = characters are preserved because they form part of the URL structure. Only the spaces are encoded.

encodeURIComponent()

encodeURIComponent() is designed to encode a single URI component, such as a query parameter value or a path segment. It encodes everything that is not an unreserved character, including the structural delimiters:

Example:

encodeURIComponent("hello world&foo=bar")

Result: hello%20world%26foo%3Dbar

Here, the & and = characters are encoded because they should be treated as literal characters within the parameter value, not as URL structure.

When to Use Each

• Use encodeURI() when you have a complete URL that might contain spaces or non-ASCII characters but whose structural characters (/, ?, &, etc.) should be preserved.
• Use encodeURIComponent() when you are building a URL piece by piece and need to encode individual query parameter names, parameter values, or path segments.

The most common pattern is building query strings:

const url = `https://api.example.com/search?q=${encodeURIComponent(userInput)}&lang=${encodeURIComponent(language)}`;

Using encodeURI() here would be incorrect because it would not encode characters like & and = that might appear in the user input.

The Space Character: %20 vs. +

You may have noticed that spaces are sometimes encoded as %20 and sometimes as +. This inconsistency has a historical explanation:

• %20 — the standard percent-encoding defined by RFC 3986. This is correct in all parts of a URL (path, query, fragment).
• + — an encoding specific to the application/x-www-form-urlencoded content type, defined in the HTML specification. This format is used when HTML forms are submitted with the GET or POST method.

When an HTML form is submitted, the browser encodes the form data using the application/x-www-form-urlencoded format, which encodes spaces as + instead of %20 for historical reasons (it predates RFC 3986). The server's form parser knows to decode + back to spaces.

In JavaScript, encodeURIComponent() always produces %20 for spaces. If you need the + encoding (for compatibility with certain APIs that expect form encoding), you can add a replace step: encodeURIComponent(str).replace(/%20/g, '+'). The URLSearchParams API in modern browsers handles this automatically.

URL Encoding in Different Languages

Every major programming language provides URL encoding functions, though the names and behaviors vary:

• JavaScript: encodeURI(), encodeURIComponent(), new URLSearchParams()
• Python: urllib.parse.quote(), urllib.parse.quote_plus(), urllib.parse.urlencode()
• PHP: urlencode() (encodes spaces as +), rawurlencode() (encodes spaces as %20)
• Java: URLEncoder.encode() (uses + for spaces), URI class for RFC 3986 compliance
• C#/.NET: Uri.EscapeDataString(), HttpUtility.UrlEncode()
• Go: url.QueryEscape(), url.PathEscape()
• Ruby: URI.encode_www_form_component(), CGI.escape()

Be aware that some of these functions use the form-encoding convention (spaces as +) while others use RFC 3986 percent-encoding (spaces as %20). Check your language's documentation to ensure you are using the correct function for your context.

Double Encoding: A Common Pitfall

Double encoding occurs when a string that has already been URL-encoded is encoded again. This turns %20 into %2520 (the percent sign itself gets encoded as %25). The resulting URL will not decode correctly because the server will decode %2520 to %20 (a literal percent-two-zero) instead of a space.

This commonly happens when:

• A URL is encoded by application code and then encoded again by a library or framework.
• User input that already contains percent-encoded characters is blindly encoded again.
• A URL is passed through multiple middleware layers that each apply encoding.

To avoid double encoding, encode each URL component exactly once, as late as possible in your code. If you are not sure whether a string is already encoded, decode it first and then re-encode it. Better yet, use the URL and URLSearchParams APIs in JavaScript, which handle encoding automatically and correctly.

Security Implications of URL Encoding

Improper URL encoding can lead to security vulnerabilities:

• Path traversal attacks: An attacker might use encoded sequences like %2e%2e%2f (which decodes to ../) to access files outside the intended directory. Servers must normalize and validate paths after decoding.
• Cross-site scripting (XSS): If user input is included in a URL without proper encoding and then embedded in HTML without proper escaping, an attacker could inject malicious scripts. Always encode URL components and HTML-escape when rendering in the browser.
• HTTP header injection: Unencoded newline characters (%0d%0a) in URLs that are used to construct HTTP headers can allow header injection attacks. Always validate and sanitize URL components.
• Open redirect vulnerabilities: Encoded URLs in redirect parameters can be used to redirect users to malicious sites. Always validate redirect targets against a whitelist.

The general rule: always encode data when constructing URLs, and always validate and sanitize URLs received from untrusted sources.

Modern JavaScript: The URL and URLSearchParams APIs

Modern browsers and Node.js provide the URL and URLSearchParams APIs, which handle URL encoding correctly and automatically:

const url = new URL('https://example.com/search');

url.searchParams.set('q', 'Tom & Jerry');

url.searchParams.set('lang', 'en');

console.log(url.toString());

Output: https://example.com/search?q=Tom+%26+Jerry&lang=en

The URLSearchParams API uses the application/x-www-form-urlencoded format (spaces as +), which is appropriate for query strings. It handles encoding and decoding automatically, eliminating the risk of double-encoding or forgetting to encode.

For path segments that need encoding, use encodeURIComponent():

const path = `/users/${encodeURIComponent(username)}/profile`;

These modern APIs are the recommended approach for working with URLs in JavaScript. They are safer, clearer, and less error-prone than manual string concatenation with encoding functions.

URL Encoding vs. Other Encoding Schemes

It is important not to confuse URL encoding with other encoding systems:

• HTML encoding (entity encoding) converts characters to &, <, ' etc. for safe inclusion in HTML documents. Different purpose, different syntax.
• Base64 encoding converts binary data to an ASCII string using a 64-character alphabet. It is used for embedding binary data in text contexts. Base64-encoded strings may themselves contain characters that need URL encoding (specifically +, /, and =), which is why base64url (a URL-safe variant) exists.
• JSON encoding escapes characters using backslash sequences (\", \\, \n, \uXXXX) for safe inclusion in JSON strings.
• Unicode escaping represents characters as \uXXXX (JavaScript), \xNN (Python), or similar language-specific sequences.

Each encoding system is designed for a specific context. Using the wrong encoding can lead to display errors, parsing failures, or security vulnerabilities. Always match the encoding to the context where the data will be used.