The PHP Execution Plugin is a Wordpress plugin which allows users to write php code inside of their posts and pages. Embedded php code will be executed when the post is shown. In doing so, the plugin offers the possibility to utilize dynamic content inside of posts.

Probably you might say, that there are already plugins that do the same!
So why another plugin that executes php code?

Well this plugin integrates well with Wordpress’ visual editor as the picture above shows. So in contrast to other plugins, you don’t need to turn off the visual editor in order to edit your php code.

Demonstration

With this plugin installed, you can insert php code into your post in html view and switch to visual view without losing or destroying the code:

This is how the php code above looks in the visual editor:

You can even drag and drop your code snippets around:

The executed php code inside the post will look like this:

Installation

This plugin is installed like any other Wordpress plugin:

• Download the latest version of the plugin.
• Unzip it into the /wp-content/plugins/ folder of your Wordpress installation resulting in a /wp-content/plugins/php_execution/ folder.
• Login as administrator and activate the plugin.
• Wordpress’ tag balancing has to be turned off. This is true by default. If you have enabled tag balancing, turn it off again by unchecking “WordPress should correct invalidly nested XHTML automatically” in Settings»Writing of the admin section.
• Optionally set PHP execution permissions in the Settings»PHP Execution panel. By default all administrators possess the permission to execute php code.
• As javascript code is added to the editor, you have to clear your browser cache !!! Otherwise the plugin won’t work as the code added by it simply won’t get executed. For information on how to clear the browser cache, follow this link.

For information on how to install Wordpress plugins in general follow this link.

Requirements

• WordPress 2.5 or higher
• PHP Execution is compatible with WordPress MU, as long as you install it in conjunction with another plugin called Unfiltered MU (http://wordpress.org/extend/plugins/unfiltered-mu/). See the comments below written by Ray for further information.

tested on WordPress 2.5, 2.6, 2.7 & 2.8 with IE 6 & 7, Firefox 3

Usage

Editing PHP

• php code can be edited in the html view of the editor.
• php code is embedded into a post in the same way as in any other html page: a php block begins with <?php and ends with ?>. The short open tag, i.e. <?, is not supported.
• One restriction is, that you are not allowed to use the ending delimeter (?>) somewhere inside a string in your php code block. If you do so, this ending delimeter will be matched and your code will inevitably break apart. So don’t write e.g.:

  some text
  <?php $test = "hello ?>"  ?>
  more text

  Writing $test = "hello ?>" instead will not lead to any problems.

• Php code is not executed in the global scope. If you need to gain access to variables in the global scope you need to “import” them first with global $var1, $var2;.
• This plugin does not evaluate every single code snippet, but the content of the post as a whole. So the following lines won’t result in errors:

  <?php if ($test==true) { ?>
  The test was successful.
  <?php } else { ?>
  The test failed.
  <?php } ?>

• When switching between html and visual view the php code is not altered in any case. All html tags, whitespaces etc. in your code are preserved.

Admin Section: setting permissions for code execution

As executing php code is a sensitive task and possible exploitation should be prevented for security reasons, the PHP Execution Plugin provides an administration panel (Settings»PHP Execution) where you can grant or deny php execution permissions to certain Wordpress roles in general or set explicit permissions for specific users.

The plugin also automatically prevents users with no php execution rights from editing posts or pages of users with these rights. It thus fixes a possible security leak where people without php execution rights can still execute code with editing other users’ posts.

ToDos

• Internationalization

License

Creative Commons GNU GPL 2.0

Download

php_execution.zip (version 1.0.1 – 2009/08/07)
browse the contents of php_execution.zip

What we’ve got here is a FileUpload component for Flex which utilizes the new FileReference.load method available since Flash Player 10 to support uploading of large files.
As a lot of providers choose a very small size for their PHP ini settings of upload_max_filesize and post_max_size, uploading files bigger than 2mb is often not supported. This can be very annoying if you like to upload mp3 or video files, which easily can exceed the maximum accepted file size. In this case you normally have to go with an FTP client.

Uploading large files

If supported by the client’s Flash Player this upload component utilizes the new FileReference.load method to load the file into the ram of the client computer at first. Once loaded the file data can be read via the data property of the FileReference object. It is now possible to extract chunks of the loaded file data and upload those chunks to the server separately. This way the upload-data won’t exceed the maximum accepted post data of the server as the chunks used are a lot smaller. On the server side, a php script is joining the chunks together to the original file.

Resuming Uploads

Prior to uploading the uploader also calculates an adler32 checksum of the file data. This checksum is used to check if a partially uploaded file already exists on the server and an upload operation can be resumed.

Screenshots

Since i don’t want unkown people to upload random files to my server, i can only offer some screenshots of the application here. The best thing would be, if you grab the source code which is available at the bottom of this page and test it out yourself.

selected files for upload

file already exists dialog

loading file into ram to for chunkwise upload

calculating an adler32 checksum of the file data to support the resume of uploads

uploading the file in chunks

License

Creative Commons GNU LGPL 2.1

Download

FlexFileUploadComponent.zip (version 0.1.4)
browse the source code and contents of FlexFileUploadComponent.zip

The following post is a copy of a post i originally contributed to the red5 mailing list.

Hello Red5 Community,

i am kind of new to red5 and spend the last days tyring to figure out how to use shared objects with red5.
In the following lines i like to summarise my experiences. This should serve as a hint for others who come accross the same questions and problems that i did.
Please feel free to correct or supplement the following lines where neccessary.
When discussing SharedObjects in the following, i refer to Remote Shared Objects and not Local Shared Objects of course. I also focus on non-persistent ones.

What are SharedObjects good for?

SharedObjects (SOs) are helpful to synchronize data between multiple clients in real-time and to invoke methods on multiple clients at once. A Remote SharedObject is an Object which is shared between multiple clients, namely all clients of the same scope.

So what is a Scope?
You can think of a scope as a chat room which is defined by the uri that is used to connect the server:

rtmp://localhost/red5test/lobby

red5test is the application scope
lobby is a room scope, a child scope of red5test
for more information on scopes see: http://jira.red5.org/confluence/display/docs/Scopes+and+Contexts

How to connect to SOs (clientside)?

A Client can connect to a SO using the following Actionscript lines:

// at first you need to establish a connection to the server of course
nc = new NetConnection();
nc.connect( "rtmp://localhost/red5test/", true );

// when the NetConnection is successfully established
// (you have to listen to NetStatusEvent.NET_STATUS event)
// you can connect to a remote SharedObject e.g. named "chat"
so = SharedObject.getRemote("chat", nc.uri, false);

// when the connection to the SO is successfully established,
// you can begin to specify its contents
so.setProperty("message","hello to all");

How does red5 handle SOs?

If a client connects to a SO, red5 looks into the specified scope if a SO named e.g. “chat” already exists.
If it does, red5 uses it, if not, red5 creates a new SO for the Scope and uses that.
When the last client connected to a SO disconnects from it, the SO will be destroyed (at least for non-persistent ones).

SOs on the serverside

a) Access SOs on the serverside
SOs can also be accessed from the serverside, i.e. the server also can make changes to the SO and broadcast them to all connected clients.

To check wether a certain SO exists you can use the following lines in your Java Application:

//get scope of the current connection first
IScope scope = Red5.getConnectionLocal().getScope();
// get SO
ISharedObject so = getSharedObject(scope, "chat");

b) Create SOs on the serverside
You can also create SOs on the serverside with the following lines:
extracted from Joachim Bauchs tutorial:

To create a new shared object when a room is created, you can override the method roomStart in your application:

public class SampleApplication extends ApplicationAdapter {

  public boolean roomStart(IScope room) {
      if (!super.roomStart(room))
          return false;

      createSharedObject(room, "sampleSO", true);
      ISharedObject so = getSharedObject(room, "sampleSO");

      // Now you could do something with the shared object...

      return true;
  }

}

If a shared object should be created for connections to the main application, e.g. rtmp://server/application, the same must be done in the method appStart.

So what is it good for to create SOs by yourself when red5 does it for you automatically?

One advantage is, that you can attach a Listener to the SO which is notified when a new client connects to the SO, updates data, disconnects etc. (See ISharedObjectListener)

Problem: clientside initiated vs. serverside initiated SOs

Creating SOs in your java application code can leed to some unexpected problems (It was at least for me).
The problem is related to something i refer to as clientside vs. serverside initiated SOs.
Of course all SOs wil be created on the red5 server. But there is a difference who initiated the creation.

• a) a serverside initiated SO is a SO that you have created with createSharedObject in your serverside code. It is then waiting for clients to connect.
  When a client connects to this SO, your custom implementation will be used.
• b) a clientside initiated SO is a “generic” SO that red5 automatically creates right in the moment when red5 recognizes that the SO a client connects does not exist.

The problem arises when you want to create a custom serverside SO for the main application:
e.g. I want to create an SO named “chat” in the main application scope and add a listener to it. So i put the following lines in my appStart method:

createSharedObject(scope, "chat", false);
ISharedObject so = getSharedObject(scope, "chat");
ISharedObjectListener listener = new MyCustomListener();
so.addSharedObjectListener(listener);

Now when i start the server, appStart is called once the Application starts. My custom SO named “chat ” will be created (a serverside-initiated SO). Now when a client connects to the SO named “chat” in the main application scope, my custom SO will be used by red5. Everything works fine untill now.
But when all clients disconnect form this SO, my custom, serverside initiated SO will be destroyed by red5. After that, when a client connects to the SO named “chat”, red5 won’t find it anymore and will automatically create a new “genric” one. This new one now is clientside initiated, as i did not create it in the serverside code with createSharedObject. The difference now is, that no listener is attached to the new generic SO, and so no events will be fired anymore.

This problem is not that severe for SOs in a room scope. In this case you put your creation code inside the roomStart method. As the room is also destroyed when the last connected client disconnects, the roomStart method will be called multiple times during the application lifecycle, i.e. everytime the first client connects to a room. In this case the problem only arises when the last client disconnects from the SO but not the room.

So what solution is there to this problem?

It would be good to have something like a SharedObjectCreationHandler or a SharedObjectFactory, but unfortunately there is nothing like that. But there is a Handler which handles sort of SO creation events: the ISharedObjectSecurity.
This handler can be registered in the Application by registerSharedObjectSecurity. Its use is to controll access to shared objects. It contains an event method named isCreationAllowed: Prior to automatically creating a SO red5 checks if creation is allowed by calling this method.
Now if you want a custom serverside initiated SO to be created everytime red5 automatically tries to create a SO, you can put your SO creation code inside this method and return true. Red5 then knows creation is allowed and tries to create a SO. To do this red5f first checks if the SO already exists and !big surprise! it does, so red5 uses it and does not create one on its own.

import static org.red5.server.api.ScopeUtils.getScopeService;
...

  public class MySecurityHandler implements ISharedObjectSecurity
  {

  ...

  public boolean isCreationAllowed(IScope scope, String name, boolean persistent)
  {
    if ( "chat".equals(name) )
    {
      // get the SO creation service. Basically the following lines are exactly
      // what MultiThreadedApplicationAdapter does. You can also pass a reference
      // to your Application to this SecurityHandler and use it instead
      ISharedObjectService service = (ISharedObjectService) getScopeService(
            scope,
            ISharedObjectService.class,
            SharedObjectService.class,
            false
      );

      if( service.createSharedObject(scope, name, persistent) == true )
      {
        ISharedObject so = service.getSharedObject(scope, name);
        ISharedObjectListener listener = new MyCustomListener();
        so.addSharedObjectListener(listener);
      }
    }
    return true;
  }

  ...

}

The method described here, was also discussed in: [Red5] creation handler to ShareObject

Last but not the least: When is it better to use a SharedObject and when to use invokes?

Here is good mail discussing that topic: http://osflash.org/pipermail/red5_osflash.org/2007-August/014406.html

That’s all,
hopefully it helps somebody.

Cheers,
Nik

—————–

Further tutorials on SOs:

• What are Remote Shared Objects (RSO) for?
• http://www.joachim-bauch.de/tutorials/red5/MigrationGuide.txt#sharedobjects
• http://www.joachim-bauch.de/tutorials/red5/HOWTO-Security.txt#shared-objects
• Shared Whiteboard application in Red5

It is based on the superb script of Alessandro Crugnola from sephiroth.it and adds some modifications and improvements, as well as an extensive help provided within the mxp flash extension file.

Have a look at the demo, the help files and/or download the component here.

Links to the DragController Component on sephiroth.it:

• Post: DragController Updated
• Component: DragController

CXS is an XML application for data serialization developed by zehnet which offers the possibility of transferring complex data structures between programming environments.

The process of creating an XML representation of application data is called serialization.
The process of instantiating data from an XML representation is called unserialization.
CXS adapts PHP’s serialization structure to XML and extends this structure with some data types known from WDDX.

CXS is aimed to offer a representation of data structures which is compact (i.e. few bytes, little overhead) and effectively to parse in different programming environments.
The idea for CXS was to build up a serialization scheme for Flash (client) – PHP (server) communication, which can be handled easily in both environments and keeps traffic as low as possible.

CXS is offered under GNU Lesser General Public License and can be used freely.

Syntax

general characteristics of the syntax:

• Every XML-element-name or XML-attribute-name has to be one ASCII-character (byte).
• Allowed ASCII-characters are the letters a-z (ASCII: 97-122). That makes 26 possible names.
• XML-element-names and XML-attribute-names are NOT handled case sensitive.
  A name therefore has to satisfy the following regular expression /[a-z]/i
• An XML-element is allowed to contain either children or character data, not both.
• child-elements of the serialized datatypes hash und object have to be parsed in a two step interval as key-value-pairs.

Data Types

CXS supports the following data types:

• scalar types: string, integer, double (aka. float), boolean, date-time
• compound types: array (numerical), hash (associative array), object
• special types: null
• resource types: binarystream

Due to the syntax rules described above, every data type is associated with a single ASCII character.
The following is a list of each data type, its associated character and the resulting XML representation in CXS.

Data Types and Serialization (CXS Packets)

boolean

a boolean expresses a truth value. It can be either true or false and has to be serialized as 1 or 0.

<b>1</b> // for true
<b>0</b> // for false

string

strings can be of arbitrary length and have to be UTF-8 encoded.

<s>hello world</s>

null

null specifies that a variable has no value or is undefined. Languages that do not have the concept of a null value should deserialize nulls as empty strings.

<n/>

date-time

date-time values have to be serialized according to ISO-8601 in the following format:
Y Y Y Y – M M – D D T H H : M M : S S + H H : M M
( Y = year, M = month, D = day, H = hour, M = minute, S = second, T = separator of date and time )
the last 2 HH indicate the timezone offset from UTC which has to be prefixed by a + (positive) or – (negative) to indicate the deviation direction.
A leading 0 is not obligatory for months, days, hours, minutes or seconds encodings.

<t>2000-02-15T09:30:25+01:00</t>

array

an array is a numerically (integer) indexed list of data elements, usually with a starting index value of 0.
Arrays will be serialized as one parent array XML element containing the list of data elements as CXS serialized children.

Example 1: The following array

$test = array( 'value1', 'value2', 3 );

will be serialized to the CXS packet

<a>
   <s>value1</s>
   <s>value2</s>
   <i>3</i>
</a>

Some programming languages demand each list element to be of the same data type.
If an array matches this restriction the flag attribute t=”[type]” can optionally be added to the CXS representation of the array. The value of the attribute t has to be the CXS type indicator (s for Strings, i for Integers, …)

Example 2: As the following array consists of elements which are all of the same type (integer), the attribute t=”i” can be added.

$test = array( 1, 2, 3 );

can be serialized to the CXS packet

<a t="i">
   <i>1</i>
   <i>2</i>
   <i>3</i>
</a>

hash

a hash is also known as associative array and is a string indexed list of data elements.
Hashes are serialized as key-value-pairs. Each hash element will result in one key and one value XML element.

Example 3: The following hash (associative array)

$test = array( 'var1' => 'value1', 'var2' => 'value2' );

will be serialized to the CXS packet

<h>
   <s>var1</s>
   <s>value1</s>
   <s>var2</s>
   <s>value2</s>
</h>

As some programming languages (e.g. PHP) do not strictly separate between array and hash, arrays can also be serialized in the CXS hash serialization scheme. CXS parsers could offer the option to define whether a strict differentiation should occur or not.

Example 4: The following array of example 1

$test = array( 'value1', 'value2', 3 );

will be serialized to the CXS packet

<h>
   <i>0</i>
   <s>value1</s>
   <i>1</i>
   <s>value2</s>
   <i>2</i>
   <i>3</i>
</h>

object

objects are serialized like hashes (associative arrays) except that the class name is stored as attribute n (name).
If a CXS parser is deserializing an object providing the name attribute it should look for a class with this name and instantiate the named object with the deserialized values if found. Anonym / Standard objects will be serialized without this attribute.

Example 5: The following php object $myClass

class test {
   var $var1 = 'value1';
   function test(){
      $this->var2 = 'value2';
   }
}
$myClass = new test();

will be serialized to the CXS packet

<o>
   <s>var1</s>
   <s>value1</s>
   <s>var2</s>
   <s>value2</s>
</o>

if the name attribute is provided deserializing would result in a new instance of test.

<o n="test">
   <s>var1</s>
   <s>value1</s>
   <s>var2</s>
   <s>value2</s>
</o>

binary

the binary datatype represents strings of binary data. Binary data has to be base64 encoded. This will ensure compatibility with general XML UTF-8 encoding.

Example 6: The php file resource

$file_resource = fopen('myFile', "r");
$CXS->serialize($file_resource);

will e.g. be serialized to the CXS packet

<c>Tm9ydG9uIEFudGlWaXJ1cyBoYXQgZm9sZ2VuZGV</c>

CXS Envelope

The CXS envelope surrounds CXS serialized data and indicates that all child nodes are CXS serialized.
If CXS serialized data is part of an XML document containing other data besides CXS, a parser should search for a CXS envelope and start parsing its children.
The CXS envelope is an XML-element with the name cxs and one attribute indicating the CXS version used for serialization. This attribute is obligatory.

<cxs v="1.1">
  <!-- CXS serialized data as child -->
</cxs>

If the whole XML document is CXS, you can omit the envelope and simply use a single CXS packet instead.

Document Type Declaration (DTD)

The following DTD can be used to validate CXS serialized data:

  <!DOCTYPE cxs [
     <!ELEMENT cxs (a|h|o|s|b|i|d|t|n|c)>
     <!ATTLIST cxs
               v CDATA #FIXED "1.1">
     <!ELEMENT s (#PCDATA)>
     <!ELEMENT b (#PCDATA)>
     <!ELEMENT i (#PCDATA)>
     <!ELEMENT d (#PCDATA)>
     <!ELEMENT t (#PCDATA)>
     <!ELEMENT n EMPTY>
     <!ELEMENT a (a|h|o|s|b|i|d|t|n|r|c)*>
     <!ATTLIST a
               l CDATA #IMPLIED
               f CDATA #IMPLIED>
     <!ELEMENT h (a|h|o|s|b|i|d|t|n|r|c)*>
     <!ELEMENT o (a|h|o|s|b|i|d|t|n|r|c)*>
     <!ATTLIST o
               n CDATA #IMPLIED>
     <!ELEMENT c (#PCDATA)>
  ]>

This tutorial covers Unicode and its UTF-8 mapping standard. It will start at zero with explaining bits and binary structures, followed by an explanation of the ASCII, extended ASCII and Unicode character sets and ending with some conclusions on how to use UTF-8 en-/decoding with Flash & PHP.

What are bytes and bits?

Bit means “binary digit” and is the smallest unit of computerized data. A bit is a 2-base number, i.e. it has either the value of 0 or 1.
A byte is an amount of memory, a certain collection of bits, originally variable in size but now almost always eight bits. This makes 2⁸ or 256 possible values for a byte.

Some example bytes could be 00000001 or 11111111 or 01010011.
Now how can we calculate the decimal value of this binary encoded byte. What we need is a conversion from base 2 to base 10.
Every 1 or 0 of these binary values is associated with an exponential of 2. For 8 bits it looks like the following:

The calculation of the decimal equivalent of the binary value 00000001:

The calculation of the decimal equivalent of the binary value 11111111:

The calculation of the decimal equivalent of the binary value 01010011:

What is ASCII?

ASCII stands for American Standard Code for Information Interchange and is a standard for assigning numerical values to the set of letters in the Roman alphabet and typographic characters. The ASCII character set can be represented by 7 bits. This makes 2⁷ or 128 different values resp. characters.
As ASCII uses only 7 of the 8 bits available of an byte the first bit is always 0: 0xxxxxxx;
Below there is a table of decimal values, their binary expressions and the character assigned to that value due to the ASCII standard. The first 32 characters are control characters.

What is extended ASCII?

“Because the number of written symbols used in common natural languages far exceeds the limited range of the ASCII code, many extensions to it have been used to facilitate handling of those languages. Foreign markets for computers and communication equipment were historically open long before standards bodies had time to deliberate upon the best way to accommodate them, so there are many incompatible proprietary extensions to ASCII.

Since ASCII is a 7-bit code, and most computers manipulate data in 8-bit bytes, many extensions use the additional 128 codes available by using all 8 bits of each byte. This helps include many languages otherwise not easily representable in ASCII, but still not enough to cover all languages of countries in which computers are sold, so even these 8-bit extensions had to have local variants.”

“Eventually, ISO released its ISO 8859 standards describing its own set of 8-bit ASCII extensions. The most popular was ISO 8859-1, also called ISO Latin1, which contained characters sufficient for the most common Western European languages. Variations were standardized for other languages as well: ISO 8859-2 for Eastern European languages and ISO 8859-5 for Cyrillic languages, for example.”
(source link: answers.com)

Often the term ASCII is associated with the 8 bit ISO 8859 standards. For the rest of the document i will refer to the 7 bit standard as ASCII and to the 8 bit ISO 8859 standards as extended ASCII.

What is Unicode?

Unicode is the attempt to create a single unified character set. Unicode is a means to assign a unique number for all characters used by humans in written language. The Unicode standard is basically nothing but tables listing every charcter in written language with its corresponding unique number. One do not need to say that this amount of characters by far exceeds 256 and one byte will not suffice to represent all these characters. Several mechanisms have therefore been suggested to implement Unicode, i.e. to represent these unique numbers as a sequence of bytes.

These mapping methods are called the UTF (Unicode Transformation Format). Among them are UTF-8, UTF-16, UTF-32. The numbers indicate the number of bits in one unit. So UTF-8 uses 8 bits per unit. This corresponds to the bits used by one byte as described above and is therefore best processed by our operating system. Hence UTF-8 emerged to the standard encoding for interchange of unicode text with UTF-16 and UTF-32 being used mainly for internal processing.

What is UTF-8?

UTF-8 is a mapping method to represent all Unicode characters as a sequence of bytes.
UTF-8 has the following properties:

• Unicode characters U+0000 (00000000) to U+007F (01111111) are encoded simply as bytes 0×00 to 0×7F (ASCII compatibility). This means that files and strings which contain only 7-bit ASCII characters have the same encoding under both ASCII and UTF-8. This makes UTF-8 a superset of ASCII.
• All Unicode characters >U+007F are encoded as a sequence of several bytes, each of which has the most significant bit set. Therefore, no ASCII byte (0×00-0×7F) can appear as part of any other character.
• The first byte of a multibyte sequence that represents a non-ASCII character is always in the range 0xC0 (11000000) to 0xFD (11111101) and it indicates how many bytes follow for this character. All further bytes in a multibyte sequence are in the range 0×80 (10000000) to 0xBF (10111111). This allows easy resynchronization and makes the encoding stateless and robust against missing bytes.
• All possible 2³¹ Unicode codes can be encoded.
• UTF-8 encoded characters may theoretically be up to six bytes long, however 16-bit BMP characters are only up to three bytes long.
• The sorting order of Bigendian UCS-4 byte strings is preserved.
• The bytes 0xFE (11111110) and 0xFF (11111111) are never used in the UTF-8 encoding.

(source link: www.cl.cam.ac.uk)

So what does all this mean? As we have learned above ASCII characters can be represented by 7 bits. Hence a 8 bit representation of ASCII characters will always have a leading 0 (0xxxxxxx). At this point UTF-8 comes in. UTF-8 encodes all ASCII as is. There will be no change. To indicate that a character is NOT part of the ASCII charset the leading 0 will be changed to an 1. So an UTF-8 character which is not part of ASCII will always start with a 1.

Furthermore the first byte (8 bits) of a byte sequence used to repesent a Unicode character also indicates how many bytes of this sequence will follow. E.g. 110xxxxx: the first 1 indicates that it is not an ASCII character, the second 1 indicates that the next byte belongs to the encoding of this unicode character. 1110xxxx: again the first byte indicates that it is not an ASCII character, the following two 1 indicate that the next two bytes belong to the encoding of this Unicode character. And so on.

The following table makes it clear:

UTF-8 example

The first thing that becomes clear is that you cannot analyze a string wether it is encoded in UTF-8 or ISO 8859. No check can be done to find out if a string is already UTF-8 encoded or not. Well you can check if the string is valid UTF-8 and hence assume it is UTF-8. A function which does that can be found in the comments to the utf8_encode in the php manual. Validating UTF-8 takes processing time and the result leaves room for discussion as it could also be an accidently valid ISO 8859 string. This string would mostly consist of some weird signs because then they are all part of the extended ASCII charset (starting with a 1).
But again, you never can be sure.

Let’s make an example:

Take a look at the following string: “© by München”.
München is the german word for Munich. This string contains two characters which are not part of the ASCII charset: the © and the ü.
These characters are part of the extended ASCII charset ISO 8859-1. As this string is not yet encoded to UTF-8 every character is represented by one byte. So if we encode this string to UTF-8 all bytes/characters of the ASCII charset will remain the same, but the © and the ü are not part of ASCII but of extended ASCII and will therefore be transfered to its UTF-8 equivalent.

Let’s take a look at the following table:

The character ü has the ISO 8859-1 value of 252. Transferred to binary this is 11111100. Now when we encode this character to UTF-8 it will be represented by two bytes 11000011 and 10111100 or as decimal values 195 and 188.

Back in ISO 8859-1 the character 195 is interpreted as à and the character 188 is interpreted as ¼.
Every UTF-8 encoded character can be interpreted by ISO 8859-1. The difference is only the perspective. As in UTF-8 a single character can consist of multiple bytes, in ISO 8859-1 every character is only one byte long. Hence the UTF-8 encoded ü will be interpreted as ü.

You can proove this by looking at the following box: It lists the UTF-8 encoded ü two times. On the left escaped as html entity and on the right the original UTF-8 encoded value. Now change the encoding for this page in your browser to ISO 8859-1 and see what happens. The right part will change to ü.

ü = ü

What happens if you reencode the already UTF-8 encoded string is that the à and the ¼ will be interpreted as ISO 8859-1 characters and will be transfered to their UTF-8 equivalents, which will then look like this: ü.

Using PHP’s utf8_encode and utf8_decode with Flash

Now let’s get to the question about if and when to use PHP’s UTF-8 functions (see php documentation: utf8_encode ; utf8_decode). The first thing we have learned is that if we have a string containing only ASCII characters, no UTF-8 encoding is needed as ASCII is a subset of UTF-8. The moment characters from the extended ASCII charset are intergrated UTF-8 encoding has to be done.

But what is with decoding from UTF-8. Let’s take a closer look on PHP’s utf8_decode function. The function description in the PHP manual is as follows: “Converts a string with ISO-8859-1 characters encoded with UTF-8 to single-byte ISO-8859-1″
This means that UTF-8 encoded data is transfered to ISO-8859-1. Now you can decide if that’s really what you want.

The big advantage of UTF-8 is that it can represent characters which ISO-8859-1 cannot. So decoding to ISO-8859-1 simply means losing these characters. They will be converted to a question mark by the utf8_decode function.

What about Flash?

As of version 6, Flash supports Unicode. Flash uses Unicode (if not specified differently with System.useCodepage) as standard for handling character data. The Unicode characters are mapped with UTF-8. (see Macromedia technote ans supportnote on Unicode support)

You often read in tutorials about Flash and the handling of special characters that you should simply use utf8_encode on strings before sending them to flash and use utf8_decode when retrieving data from flash and everything will be fine.
This is simply not true.

Let’s take a deeper look on this case.

Let’s assume the charset your server uses is ISO-8859-1. Using utf8_encode on your data before sending it to flash is mostly a good idea namely in the case that your data is not already UTF-8 encoded and is containing characters other than ASCII. If you are sending only ASCII characters you don’t need to encode the string to UTF-8, but if you do so it will be no problem.

What will happen if you receive data from Flash and use utf8_decode?

All UTF-8 characters will be converted to ISO-8859-1. This means the ü from the example above will be converted to its ISO-8859-1 equivalent.But what if the person sending the data e.g. through an Flash inputfield is from Russia, Japan etc. In this case there will be no ISO-8859-1 equivalent for the UTF-8 characters and the characters get replaced by a question mark. This is apparently not the desired result.

The solution for this is quite simple. If you are planning a flash guestbook, forum, chat etc. which should support all languages just keep your fingers off the utf8_encode and utf8_decode function. Just save your received data as is to e.g. a database. The result will be some weird signs as they are interpreted as ISO-8859-1. But don’t care. When it’s time to send this data back to flash, get your data from the database and send it as is, because it is already UTF-8 encoded by Flash. All other data which does not originate from Flash, e.g. some text for a button, has to be UTF-8 encoded (again only if it contains characters of the extended ASCII charset) before it is sent to Flash.

What about textfields in Flash?

You must not use textfields with embedded fonts. As these textfields will only show the characters you have embedded and you cannot embed the whole Unicode charset. If fonts are not embedded Flash will look on the client’s system for the font you chose and will use this font to display the content of the textfield. In this case the restriction is on the font’s supported characters. To display most Unicode characters on Windows systems the font Arial Unicode MS is the best choice, as it is Windows standard and supports 51,180 characters.

Another possibility is using Flash’s device fonts. Flash’s device fonts are no fonts themselves. A device font defines a general font type. During playback of the movie, these fonts will be substituted with an installed font that matches that type. E.g. the device font _sans will normally be mapped with Arial on Windows systems and Helvetica on a Mac. These mapped fonts normally are standard system fonts with a good chance to support Unicode characters. See this Macromedia technote for further information.

CXS and UTF-8

Now when using a CXS parser on data already UTF-8 encoded you can use the flag NO_UTF8 to prevent encoding. On the other side when retrieving data from Flash which possibly consists of characters which are not in the ISO-8859-1 charset use the flag NO_UTF8 to prevent decoding to ISO-8859-1.

Links

Some links with additional information on the topic:

• What is Unicode? by unicode.org
• UTF-8 specification ISO 10646-1:2000 Annex D and RFC 3629
• a good UTF-8 sample site
• unicode tables, an UTF converter etc. are available on this site
• a short and good tutorial on Flash MX & Unicode