Our website makes use of cookies (sadly not the delicious, crumbly ones) and similar technologies. If you accept them, we share information with our partners for social media, advertising and analysis.
Please let us know which cookies we can use.
Manage Cookies
Necessary
These cookies are required in order for our website to function (e.g. logging in). If you set your browser to block or alert you about these cookies, some parts of the website might not work.
Targeting and Advertising
Advertisers and other content providers that may appear on our website may also use cookies that are not sent by us. Such advertisements or content may use cookies to help track and target the interests of users of the website to present customised and personalised advertisements or other messages that the user might find interesting. We also use these cookies and so-called Tracking Pixels of our partners to measure and improve the effectiveness of marketing campaigns.
Social Media
These cookies enable the website to provide enhanced functionality and personalisation. They may be set by third-party providers (like social networks or streaming platforms) whose services we use on the website. If you do not allow these cookies, some or all of these services may not function properly. (YouTube)
WalletConnect
When using the Tibia Token Exchange feature on the Account Management page, the third party provider WalletConnect is used to connect to your cryptocurrency wallet. WalletConnect sets cookies to ensure the legitimacy of the application and to enable the connection to your wallet. If you do not allow these cookies, you cannot use the Tibia Token Exchange.
+
+
\ No newline at end of file
From d8740ecfd6dd30652d4da561cb6d2ceef3b098c8 Mon Sep 17 00:00:00 2001
From: Skyliife <82951416+Skyliife@users.noreply.github.com>
Date: Fri, 18 Apr 2025 16:47:27 +0200
Subject: [PATCH 2/3] fix: proper decode legacy encodings in HTML collector for
Umlauts
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
- fix for character endpoint
- Replace custom TibiaDataConvertEncodingtoUTF8 with golang.org/x/net/html/charset.NewReader
- Use the actual Content‑Type header from Tibia.com to normalize response bytes into UTF‑8
- Remove resIo/resIo2 steps and feed the UTF‑8 reader directly into goquery
---
src/TibiaDataUtils.go | 3 ---
src/TibiaDataUtils_test.go | 5 ++++-
src/webserver.go | 22 +++++++++++++---------
3 files changed, 17 insertions(+), 13 deletions(-)
diff --git a/src/TibiaDataUtils.go b/src/TibiaDataUtils.go
index 5e2903f8..b239c67f 100644
--- a/src/TibiaDataUtils.go
+++ b/src/TibiaDataUtils.go
@@ -87,9 +87,6 @@ func TibiaDataQueryEscapeString(data string) string {
// switching "+" to " "
data = strings.ReplaceAll(data, "+", " ")
- // encoding string to latin-1
- data, _ = TibiaDataConvertEncodingtoISO88591(data)
-
// returning with QueryEscape function
return url.QueryEscape(data)
}
diff --git a/src/TibiaDataUtils_test.go b/src/TibiaDataUtils_test.go
index 94552c40..713cdd88 100644
--- a/src/TibiaDataUtils_test.go
+++ b/src/TibiaDataUtils_test.go
@@ -150,16 +150,19 @@ func TestEscaper(t *testing.T) {
strOne = "god durin"
strTwo = "god+durin"
strThree = "gód"
+ strFour = "Näurin"
)
sanitizedStrOne := TibiaDataQueryEscapeString(strOne)
sanitizedStrTwo := TibiaDataQueryEscapeString(strTwo)
sanitizedStrThree := TibiaDataQueryEscapeString(strThree)
+ sanitizedStrFour := TibiaDataQueryEscapeString(strFour)
assert := assert.New(t)
assert.Equal(sanitizedStrOne, "god+durin")
assert.Equal(sanitizedStrTwo, "god+durin")
- assert.Equal(sanitizedStrThree, "g%F3d")
+ assert.Equal(sanitizedStrThree, "g%C3%B3d")
+ assert.Equal(sanitizedStrFour, "N%C3%A4urin")
}
func TestDateParser(t *testing.T) {
diff --git a/src/webserver.go b/src/webserver.go
index 4b50bb6f..b88fad42 100644
--- a/src/webserver.go
+++ b/src/webserver.go
@@ -5,6 +5,7 @@ import (
"encoding/json"
"errors"
"fmt"
+ "golang.org/x/net/html/charset"
"log"
"net/http"
"os"
@@ -303,7 +304,8 @@ func tibiaCharactersCharacter(c *gin.Context) {
TibiaDataErrorHandler(c, err, http.StatusBadRequest)
return
}
-
+ fmt.Println(name)
+ fmt.Println(TibiaDataQueryEscapeString(name))
// Build the request structure
tibiadataRequest := TibiaDataRequestStruct{
Method: resty.MethodGet,
@@ -1289,14 +1291,16 @@ func TibiaDataHTMLDataCollector(TibiaDataRequest TibiaDataRequestStruct) (string
return string(res.Body()), nil
}
- // Convert body to io.Reader
- resIo := bytes.NewReader(res.Body())
-
- // wrap reader in a converting reader from ISO 8859-1 to UTF-8
- resIo2 := TibiaDataConvertEncodingtoUTF8(resIo)
-
- // Load the HTML document
- doc, err := goquery.NewDocumentFromReader(resIo2)
+ // Decode the raw response into real UTF-8 using the Content‑Type header
+ utf8Reader, err := charset.NewReader(
+ bytes.NewReader(res.Body()),
+ res.Header().Get("Content-Type"),
+ )
+ if err != nil {
+ log.Printf("[error] TibiaDataHTMLDataCollector charset.NewReader failed: %s", err)
+ return "", err
+ }
+ doc, err := goquery.NewDocumentFromReader(utf8Reader)
if err != nil {
log.Printf("[error] TibiaDataHTMLDataCollector (URL: %s) error: %s", res.Request.URL, err)
return "", err
From b82bd71c5c9bf5cd529c090d54bf9d34d2f5856b Mon Sep 17 00:00:00 2001
From: Skyliife <82951416+Skyliife@users.noreply.github.com>
Date: Fri, 18 Apr 2025 16:50:45 +0200
Subject: [PATCH 3/3] fix: proper decode legacy encodings in HTML collector for
Umlauts
- cleanup
---
src/webserver.go | 3 +--
1 file changed, 1 insertion(+), 2 deletions(-)
diff --git a/src/webserver.go b/src/webserver.go
index b88fad42..29343ff6 100644
--- a/src/webserver.go
+++ b/src/webserver.go
@@ -304,8 +304,7 @@ func tibiaCharactersCharacter(c *gin.Context) {
TibiaDataErrorHandler(c, err, http.StatusBadRequest)
return
}
- fmt.Println(name)
- fmt.Println(TibiaDataQueryEscapeString(name))
+
// Build the request structure
tibiadataRequest := TibiaDataRequestStruct{
Method: resty.MethodGet,