| 01:05 | spunkygoose | I am just wondering how to log into my clearfog SBC console? It only has microUSB and I have never done this via this interface before? |
| 03:40 | mk01 | spunkygoose: just attach to yours pc USB port - it emulates serial |
| 04:01 | spunkygoose | ah, windows will auto-detect the usb device as a serial device? |
| 04:01 | spunkygoose | @ mk01 |
| 04:01 | mk01 | yes |
| 04:01 | mk01 | should |
| 04:01 | spunkygoose | ok, very good, and it comes up as a reguarly tty that I can use putty, etc... ? |
| 04:02 | mk01 | (don't have windows to test, but it is pl301 compatible - macosx, linux, by default have this drivers - I suppose win too) |
| 04:02 | mk01 | yes, it creates ttyUSB |
| 04:03 | mk01 | tty called ttyUSB / ttySerial on mac, I'm using screen or minicom, but putty will be OK too |
| 04:04 | mk01 | spunkygoose: don't be mislead by the "usb" connector - it is really serial as you expects |
| 05:55 | spunkygoose | @mk01 - thank you I do appreciate it! |
| 07:59 | mWaltari | ShockAbsorbereita... -50% jostain hinnasta :D |
| 08:02 | mWaltari | OOps... Wrong forum :D |
| 08:03 | mWaltari | Sorry 'bout that :D |
| 14:37 | topi` | jnettlet: is there any guide as of how to force the HB to suspend state? (to save power) -- and how can I wake it up after 60 seconds? |
| 14:37 | Fira_ | Oh my god. Guys. How did you manage to make anything useful out of the mess that is the i.MX6 ? It's been month and we're jumping from problem to problem, things becoming worse and worse as we try to demonstrate what could be wrong. |
| 14:37 | topi` | after 60sec it needs to poll the server, again |
| 14:38 | topi` | Fira_: I have a couple of i.MX6's that work just fine for me |
| 14:38 | topi` | of course, it has its fair share of bugs, but almost everything out there has |
| 14:39 | Fira_ | Well, where do I start.. |
| 14:39 | Fira_ | We knew the FEC controller is supposed to have a 470 Mbps throughput limit theorical due to bus problems |
| 14:39 | Fira_ | So we pluggued in an IGB card on our SABRESD's PCIe slot |
| 14:40 | Fira_ | When we're receiving packets, CPU is capped by overhead processing before the line rate, despite this thing having a quad-core A9 |
| 14:41 | Fira_ | We did some testing and rate is near-linear with MTU which lead us to the conclusion the whole of packet processing is going uber-slow for /some reason/ |
| 14:41 | vpeter | topi`: something like echo mem > /sys/power/state but to resume I used only serial console. |
| 14:42 | Fira_ | So like, on a hunch, i do a rough test of the onboard 800 Mhz DDR3 with a dd of /dev/zero to tmpfs. Result: 1.4 Gbps, one tenth of what we're getting on a nearby x86 with DDR2 |
| 14:43 | Fira_ | Of course this doesn't prove anything so we just keep digging' |
| 14:43 | topi` | vpeter: yes. I guess I need to set up some timer interrupt to do the wakeup |
| 14:43 | Fira_ | We tried doing a bridge between the IGB and FEC ethernet, and perf was below 200 Mbps in most test cases |
| 14:45 | Fira_ | Now as I said earlier the thing is, packet receipt seems slow and to scale off MTU, which spells to me that something is wrong with the per-packet processing time. |
| 14:45 | Fira_ | We graph it -- see weird occurences of sudden slowdowns over multiples of 2KB MTU. Proceed to investigate with Kernel's perf counters |
| 14:46 | topi` | I think 1.4 Gbps (140 MB/s) seems to be a fair number for dd'ing from /dev/zero to /dev/shm |
| 14:46 | Fira_ | I boot Freescale's official image and run perf top. => Obscure IRQ problem => Oops => CPU0 stalled => Hard-lockup |
| 14:47 | topi` | 943718400 bytes (944 MB) copied, 3.29413 s, 286 MB/s |
| 14:47 | topi` | this is from a Snapdragon 600 -based Utilite2, which has a bit faster RAM than the Hummingboard |
| 14:47 | topi` | using 1MB block size |
| 14:48 | topi` | this is not totally RAM speed bound, many core and uncore things affect that figure |
| 14:49 | Fira_ | though try on an IMX6, we're getting half that, despite the Snapdragon 600 using LPDDR3 clocked at 566 Mhz and the i.MX6Q clocked at 800 Mhz |
| 14:50 | topi` | I don't think I/O is the strong suit of the i.MX6, for better I/O there's the Marvell Armada |
| 14:50 | Fira_ | anyway to get back on perf, we boot up our buildroot vanilla 4.3 image and try to run perf on it |
| 14:50 | Fira_ | -> oops -> IRQ disable -> botched results |
| 14:50 | Fira_ | try again 80 times |
| 14:50 | Fira_ | eventually it'll work, then hard-lockup the system due to obscure interrupt load |
| 14:50 | topi` | I haven't tried the Freescale image, only the SR-provided kernel |
| 14:52 | topi` | I should test out the gigabit ethernet, I haven't really done it since a few of my devices are on the 100Mbps hub |
| 14:52 | Fira_ | yeah in fact we initially cooked our own patchset based off SolidRun's, with additional fixes and addition of I-pipe and Xenomai 3 Cobalt |
| 14:53 | Fira_ | we pretty much gave up using it now because it had just so many problems... but now we realize it wasn't just us messing up since there's perf issues all over the place, even with Vanilla and Freescale's provided images |
| 14:54 | Fira_ | anyway regarding being unable to debug and it locking up the system |
| 14:54 | Fira_ | Errata 006259 |
| 14:54 | Fira_ | "When JTAG_TCK is not |
| 14:54 | Fira_ | toggling after power-on reset (POR), |
| 14:54 | Fira_ | the ARM PMU, PTM, and ETB stay |
| 14:54 | Fira_ | in their disabled states so various de |
| 14:54 | Fira_ | bug and trace functions |
| 14:54 | Fira_ | are not available." |
| 14:55 | Fira_ | => if you don't have a debugger physically connected at the moment of bootup, you can't do software debugging later on |
| 14:56 | Fira_ | so we can't even show just how horrid the perf is because even the tracing functions don't work |
| 14:57 | Fira_ | Recap |
| 14:59 | Fira_ | - FEC has bad perf per design and Freescale acknowledges it, capping it at 470 Mbps simplex |
| 15:00 | Fira_ | - Using a PCIe external card or FEC on our systems, perf is also capped around 150-300 Mbps due a bottleneck in packet processing |
| 15:01 | Fira_ | - The little I've seen so far in terms of PCIe benchmarks of the i.MX6 indicates its throughput is also two to three times below spec |
| 15:02 | Fira_ | - Memory Throughput is way below spec and two to three times less than you'd expect from the platform |
| 15:04 | Fira_ | - Because of that, anything involving even simple fast-path processing takes ages, capping our bridging attempts near 300 Mbps depending on test cases |
| 15:05 | Fira_ | - The tracing functions to eventually prove this could or could not be a software-side problem do not work unless you start hackin' the board yourself with JTAG |
| 15:06 | Fira_ | - The FEC driver has numerous bugs and seems to regularly oops on all the images we tried |
| 15:06 | Fira_ | - FEC doesn't support Jumbo-frames, which is the only solution to poor packet processing we found so far |
| 15:08 | Fira_ | TL;DR: Perf is abysmal compared to what you'd expect from the platform components |
| 15:08 | Fira_ | I mean, what good is it selling a SoC with an A9, GigE, PCIe and such, if they can't remotely achieve specced speeds ? :/ |
| 15:09 | mk01 | Fira_: is that all valid for imx6+ too ? |
| 15:09 | Fira_ | I'm not sure |
| 15:10 | Fira_ | I overheard, but don't take that for granted, that the IMX6QP had bugs fixed regarding memory handling for example |
| 15:11 | mk01 | that's why I'm asking. + was promised to fix the design problems on internal bus etc |
| 15:11 | Fira_ | All I know is that when we told the people we're forced to run this silly project with about the possible memory throughput issues, they weren't suprised at all and mentionned that Freescale had "fixed a problem with the memory controller impacting performance by an important factor" |
| 15:11 | Fira_ | in the + version |
| 15:13 | Fira_ | Anyway, it's pretty sad that they picked some of the best components around and duct'taped them together into something that doesn't fare well at all under stress x( |
| 15:14 | mk01 | ... the AXI bus is killing pretty much everything |
| 15:15 | Fira_ | :/ |
| 15:17 | mk01 | I don't have the + to test myself, but Jnettlet was saying that the bandwidth is double/triple of what non+ had. so if there is a project running (too far to change platform), + could be a way to go |
| 15:18 | Fira_ | Yeah I guess we'll have to hammer that idea, cause right now for what we're trying to do, the standard one is totally a no-dooer :P |
| 15:20 | Fira_ | I just wish Freescale/NXP would assume they kinda screwed up with the standard version, it's very hard to find all of this and we lost a good number of months |
| 15:20 | Fira_ | Had the same problem with graphics support |
| 15:21 | Fira_ | Took weeks trying to get OpenGL to work with the last versions before I eventually understood they silently dropped DRI/OpenGL support for just OpenGL ES at some point |
| 15:22 | Fira_ | It's a thing to make mistakes or design choices, it's another to hide them :/ |
| 15:24 | mk01 | yes, I have to agree with that - although have to say this looks be the "standard" around embedded platforms (IMHO). |
| 15:24 | mk01 | unfortunately |
| 15:32 | Fira_ | I guess... had the same problem with another product of theirs too... Took two whole days searching for some missing software sources for building images, before stumbling on a single lost post in a Mailing List where an employee said they had to remove them due to licensing constraints |
| 16:23 | topi` | I wonder how long we have to wait to get MicroSOMs based on the Plus versions of i.MX6s |
| 16:24 | topi` | rabeeh: do you have any info about i.MX6 plus? |
| 16:55 | rabeeh | topi`: we have them in the lab |
| 16:55 | rabeeh | but we didn't see lots of demand in the market for them; so they are second in priority |
| 17:03 | topi` | OK |
| 17:04 | topi` | do they carry a higher price tag than the old i.MX6 models? |
| 17:04 | rabeeh | oh yeah |
| 17:05 | rabeeh | it's mainly about GPU performance topi` |
| 17:05 | rabeeh | i recall that you don't use the GPU at all; right? |
| 17:36 | KBme | rabeeh, any news about the 3.18 kernel pcie issue? |
| 17:58 | mk01 | rabeeh: heh, so it was foolish to expect FSL will simply change imx6 for imx6+ (i mean as product replacement)? |
| 18:19 | rabeeh | KBme: my brain leaked; what this issue was?| |
| 18:19 | rabeeh | mk01: they have lots of deployments in the field; they won't just wipe imx6 from existence and replace it with imx6+ |
| 18:27 | KBme | rabeeh, well, seems like some timings issues with the pci bridge is causing some bugs in the "stable" kernel |
| 18:27 | rabeeh | which device? |
| 18:27 | rabeeh | imx6? |
| 18:27 | KBme | so I have a couple of machines where the pcie wifi card works, and a lot of machines where it doesn't |
| 18:27 | KBme | yeah, this is on a hummingboard2 |
| 18:28 | rabeeh | i wish jnettlet was around |
| 18:28 | rabeeh | he banged his head with lots of imx6/pcie issues |
| 18:28 | KBme | uh oh? |
| 18:28 | rabeeh | why 3.18? |
| 18:29 | KBme | in any case, a while back I was told the software team's current prority is android so I had to wait |
| 18:29 | KBme | because of the graphics drivers? |
| 18:29 | rabeeh | which kernel are you using KBme? |
| 18:29 | rabeeh | where the 3.18 is coming from? |
| 18:30 | KBme | err |
| 18:30 | KBme | standard solid-run |
| 18:30 | KBme | gimme a sec |
| 18:30 | rabeeh | https://github.com/SolidRun/linux-fslc/commits/3.14-1.0.x-mx6-sr |
| 18:30 | rabeeh | ? |
| 18:31 | KBme | https://github.com/SolidRun/linux-fslc branch .. |
| 18:31 | KBme | yeah that one |
| 18:31 | rabeeh | ok |
| 18:31 | KBme | sorry, it's 3.14 |
| 18:31 | rabeeh | jnettlet fixed and backported lots of pcie issues |
| 18:31 | rabeeh | which cards are you using? |
| 18:31 | KBme | oh lemme see if there are now commits! |
| 18:31 | rabeeh | which wifi card are you battling with? |
| 18:32 | rabeeh | i don't have the exact list but we have lots of customers with working 11ac cards |
| 18:32 | rabeeh | and 11n ofcourse |
| 18:32 | KBme | it's dnxa-116 from I forget which vendor |
| 18:32 | KBme | but solid-run is supplying them with the boards |
| 18:32 | rabeeh | http://www.unex.com.tw/wi-fi/dnxa-116 |
| 18:33 | KBme | and I hardly have any bandwidth so I can't really do the web thing right now |
| 18:33 | KBme | happy irc still works |
| 18:33 | KBme | yeah, that must be it |
| 18:33 | rabeeh | we are supplying this card? really? |
| 18:33 | KBme | yeah |
| 18:33 | rabeeh | this must be with the operations team. |
| 18:34 | KBme | no clue |
| 18:35 | KBme | so I can't even load the github commits on that branch currently |
| 18:43 | jnettlet | I am here trying to catch up on the backlog |
| 18:44 | KBme | well, if you have made some kernel updates to that branch in the last two months then never mind me, I'll have to test the new patches |
| 18:44 | jnettlet | some cards still have PCIe issues due to NXP patches. Upstream drivers don't have the problems. |
| 18:45 | jnettlet | It seems mostly Intel based cards exhibit the problems where they are not detected |
| 18:45 | jnettlet | I assume the dnxa card has the same issues |
| 18:45 | KBme | probably, yes. |
| 18:46 | KBme | I was told something about timings, but I'm not really a hardware guy so I don't know much. |
| 18:47 | jnettlet | Fira_ all ARM hardware supports the OpenGL ES standard. People implementing OpenGL are trying to make something work the way it isn't supposed to |
| 18:58 | KBme | jnettlet, yeah, this is the same version of the kernel that I am testing, so the issue remains |
| 19:00 | jnettlet | KBme, could you test upstream 4.4 or mainline to verify it works there? |
| 19:00 | jnettlet | If you need something easy to install we can package something for you |
| 19:00 | KBme | yeah, the person I am in communication from solid-run asked the same thing and I still haven't tried it. I will, and tell you the result. |
| 19:01 | jnettlet | I just want to verify it fixes your specific case |
| 19:01 | KBme | jnettlet, well, I do need the hardware gpu acceleration |
| 19:01 | jnettlet | Okay great |
| 19:01 | KBme | so I need the 3.14 kernel |
| 19:01 | KBme | but I can add a kernel patch really easily, packaging should not be a problem at all |
| 19:02 | KBme | (using a modified openelec distribution to build) |
| 19:02 | jnettlet | Yes I understand. This is just to make sure mainline PCIe fixes your problem |
| 19:02 | KBme | yeah, I got it. |
| 19:02 | jnettlet | Then I will sort out what NXP has broken in their drivers. |
| 19:02 | KBme | err, maybe not. what is the upstream 4.4 and what is mainline? |
| 19:06 | jnettlet | KBme, you are running OpenElec? |
| 19:06 | KBme | that's right |
| 19:07 | KBme | or, well, a heavily customized version, at least. |
| 19:08 | jnettlet | Test the latest beta build. I think sraue has pushed a 4.4 based kernel. I don't recommend basing off it because it has other bugs, but may be enough for you to test from. |
| 19:09 | KBme | yeah, I saw libreELEC |
| 19:09 | KBme | so I might switch at some point |
| 19:10 | KBme | I don't use most of the extra stuff that's included in openelec anyways |
| 19:10 | jnettlet | Okay I have been working with them directly the last two weeks |
| 19:10 | KBme | and interesting about the 4.4, i'll make sure to check that out. |
| 19:10 | KBme | so mainline is the kernel from kernel.org I guess? which version should I try? |
| 19:11 | KBme | and where is the "upstream" 4.4 kernel code located at? |
| 19:11 | KBme | gotta run right now, but i'll be back soon |
| 19:11 | KBme | thanks so far jnettlet |
| 19:12 | jnettlet | I will be back in my office on Friday. Can you follow up then and we will get this figured out. |
| 19:18 | vpeter | jnettlet: sraue is using solidrun's 3.14. |
| 19:27 | jnettlet | vpeter, I thought you said he had switched? |
| 19:28 | vpeter | Yes, from old solidrun to new. |
| 19:28 | vpeter | but 3.14. |
| 19:28 | jnettlet | Do you have one of your test builds that KBme could test with? |
| 19:28 | vpeter | Not at hand. |
| 19:28 | jnettlet | Ah okay.. My mistake thanks for correcting me |
| 19:29 | jnettlet | Okay I will sort something out then |
| 19:29 | vpeter | there is on forum. Let me find. |
| 19:29 | vpeter | http://forum.solid-run.com/kodi-xbmc-player-on-cubox-i-and-hummingboard-f7/-devel-2016-04-04-libreelec-master-v7-0-kodi-16-1--t2964.html |
| 19:30 | jnettlet | Thanks. |
| 22:07 | KBme | jnettlet, sorry, something came up |
| 22:07 | KBme | I'll try with an upstream open/libreelec to see if the wifi module works |